Manipulation of text by programs, scripts, etc.
0
votes
1answer
21 views
Transform plain data into json file
everyone.
I am quite new with linux and I am learning a lot with all of you reading this site. I am into a bit of trouble with my accounting program and that's why I am recurring to you. I want to ...
-2
votes
2answers
44 views
another sed question
I have a daily CSV export of files of about 200-50 Gb per/day. Each CSV file's first field is a date information like that. Each CSV file has 22 fields
/data/exported$ head ...
0
votes
1answer
23 views
Compare two columns and insert value to next column
I have file which contains data in columnwise manner. Please help me to get desired output. I have tried to use awk but not able to insert the value.
- A B C D E
- 6 15 78 4
- 4 14 77 4 ...
2
votes
3answers
30 views
How to append some line matched to previous line matched with sed?
For example, transfer below
00:00:10.730
this presentation is delivered by the
00:00:13.230
Stanford center for professional
00:00:14.610
development okay so let's get started
00:00:25.500
with ...
0
votes
1answer
18 views
Shell script problem in extracting contents from Text file [on hold]
I have a text file called test.txt on linux box. Contents in text file is like:
Java
1 It is object oriented programming language.
2 Widely used for web based and enterprise applications.
3 Java is ...
0
votes
3answers
30 views
How can I use sed/grep/awk to print the TLD's from a list of hostnames that have the TLD in different fields?
I have a very large list of hostnames from which I am trying to print the TLD (.com, .net, .info, etc.) of each host. The problem is that the hosts have their TLDs in different fields, so I can't ...
2
votes
3answers
40 views
merging two files according to a common column
I have 2 files. File1 is:
chr19 4124051 4124250 1
chrX 154458151 154458200 2
chr22 37019451 37019600 3
chr15 74995401 74995550 4
chr12 128823901 128824100 5
and File2 ...
10
votes
5answers
375 views
How to sort columns based on the first line?
I need to sort the columns of a very big dataset (1000 lines and 700000 columns).
As an example, my columns are randomly arranged like: col1 col4 col3 col2, and I need to sort that.
I have been ...
0
votes
3answers
55 views
Using shell scripting I need to combine data from two files into one file
file 1 contains:
hour value1
01 25
02 24
03 24
04 27
05 30
file 2 contains
hour value2
01 03
03 05
...
-3
votes
0answers
18 views
how to update a column with file names in a file with the names of files generated in differect directory in unix [on hold]
how to update a column with file names in a file with the names of files generated in differect directory in unix
input file:
input.txt
1 abc_123.txt
2 vcb_965.txt
source path"
cd /etldata/.../...
-3
votes
4answers
28 views
how to reterive the value present within the brackets
I need to retrieve the number which is present in bracket only with total comamnd line.
For ex:
$ cat exa.lg
02:57:25 BROKER : Unable to open errno =. (1295)
02:57:25 BROKER 1: Unable to open (45)...
4
votes
6answers
387 views
AWK question: Print N lines, starting from the third line after given /pattern/
I am an AWK newbie.
I am producing a huge file which includes sections like this:
~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~ Gradients ~~~~~~~~
~~~~~~~~~ x y z ~~~~~~~~~~
~ ~
~ H 1 0.00781 ...
1
vote
0answers
34 views
Interpolation to fill missing data (x,y)
I have a dataset of spectral measurements (x,y), for which the middle part (y) is missing (below is a subsample, missing is a series of spectrum data from 470nm to 570 nm):
300,07 9,742
307,07 11,...
3
votes
2answers
112 views
Parse logs for certain values
I am monitoring the history of convergence of a certain problem. The history output is as follows:
Time = 24
Calculate volume forces from actuator disk
Total thrust = -8.46832
Total torque = 1.03471
...
0
votes
3answers
39 views
Parsing CSV using AWK or SED
I'm trying for many hours to clean my csv file using (AWK or SED)
here is how looks the csv file:
id,name,contact-type,contact
"1","toto corp","tel","+123456789"
"1","toto corp","fax","+198765432"
"...
0
votes
1answer
32 views
Search and replace a sequence inside bracket
I have a file in which there is a line
byte address[] = { 0x54, 0x36, 0x45 };
I am trying to change the file by 3 command line parameters provided by the user -
grep byte | sed 's/\{.*\}/{ 0x$1, 0x$...
0
votes
1answer
35 views
Classifying a log file in bash [on hold]
I have a log file that includes the time, the ip, the date and the status, and the status is either an error or a warning.
Collect each log in a separate text file according to the status, either ...
0
votes
1answer
41 views
Finding average excluding the first row
I have a csv file with many columns. I have computed average of a column using the command
awk -F',' '{sum+=$3} END {print sum/NR}' cpu.csv
But it does include the first row that has text fields ...
-3
votes
1answer
28 views
Retrieve Nth records from a file until end of file is reached [duplicate]
How can I retrieve the Nth record of a large text file and insert them into a new file?
-2
votes
4answers
63 views
Printing every Nth line out of a large file into a new file [duplicate]
I am trying to print every Nth line out of a file with more than 300,000 records into a new file. This has to happen every Nth record until it reaches the end of the file.
0
votes
2answers
51 views
Convert text file to JSON
Beginner here, I hope you can direct me to right path. Using curl, I get some statistical information from my TP-Link router. I put that information to a text file. However, I need this info in JSON ...
0
votes
0answers
30 views
Extracting text blocks based on grep output
I'm currently working with a text file that contains the following text blocks:
--------------------------------------
Beginning of block
Text
Random Text
keywordA
Text
End of block
------------------...
0
votes
2answers
36 views
Replacing characters in file
Hello I am writing script which takes 2 symbols and replaces the first with the second and also the script takes as an arguments files .The idea is to replace in the file the first character with the ...
3
votes
1answer
51 views
Colorize output from sensors
I use sensors to keep an eye on CPU temperatures on the console. This is part of the output:
coretemp-isa-0001
Adapter: ISA adapter
Physical id 1: +45.0°C (high = +80.0°C, crit = +90.0°C)
Core 0: ...
0
votes
1answer
29 views
How do I find username that in total uses the most CPU time?
How do I find username that in total (i.e sum for all processes) uses the most CPU time?
Also I want to find the total amount of memory consumed by running processes of that user and output processes ...
1
vote
1answer
25 views
how to add output as a new column with the file names
How can I add output of each file incrementally in one singel output? I want to do this instead of running paste command on all files together. It is because I have 10k files and each file is 100 GB ...
3
votes
3answers
55 views
How to pass a string that specifies columns to be printed to awk?
I have a file with a large number of space separated columns. I want to print specific columns based on certain numerical criteria in a dynamic way. For example:
]$ cols=$(for i in `seq 1 3`; do echo ...
0
votes
3answers
48 views
Mixed uppercase and lowercase in a word
I have a list of words and I want to make one character lowercase and one character uppercase. For example, the word information I want to be
iNfOrMaTiOn
and
InFoRmAtIoN
0
votes
3answers
32 views
Extract third group of text from range delimiters
I have a file.txt that contains the single line:
[MOVING] From [/source/foo.txt] to [/dest/bar.txt]
I would like to find a regex that basically extract the third group inside a pair of square ...
2
votes
3answers
106 views
Remove spaces except lines beginning with a specific pattern
I want to remove all spaces from a file, except from every line beginning with the same pattern (pattern is "ORGANISM").
Input:
Cat; Dog; Squirrel
ORGANISM Animalus terrus
Sequence: ...
1
vote
2answers
40 views
Modify records in fixed-width files
I have multiple fixed width files with the structure like this:
datafile3248.dat
HEAD
DESCRIPTION
NAME 1 A 8 X
NAME 2 A 8 X
NAME 3 A 9 XX
NAME 4 A 9 XX
NAME ...
1
vote
1answer
70 views
sed code understanding for text processing
Could anyone explain me this below sed code
sed -n '
/Policy Name:/! d
s/.*:\s\+//
h
:1
n
/Active:\s*no/d
/HW\//!b1
:2
s/.*\s\(\S*\)\s*/\1/
G
s/\n/\t/p
n
...
0
votes
3answers
42 views
How to match a pattern in lines before another pattern match
This question is similar to How to show lines after each grep match until other specific match?
I want to match a particular pattern in lines before another pattern match.
Here I want to get the ...
8
votes
9answers
646 views
Change the order of lines in a file
I'm trying to change the order of lines in a specific pattern. Working with a file with many lines (ex. 99 lines). For every three lines, I would like the second line to be the third line, and the ...
0
votes
4answers
71 views
How to join rows with single columns to a maximum of 4 columns in one row?
I have a file looks like below:
1
4 5 6 7 19
20
22
24 26 27
29
30
31
32
34
40
50
56
58
100
234 235 270 500
1234 1235 1236 1237
2300
2303
2304
2307
2309
As it is clear there are some rows with ...
1
vote
2answers
38 views
Compare columns of different files with different sizes and replace with NA non matching values
I have three data frame:
Dataframe 1
chr start end Id
chr1 1 400 SN_1
chr1 401 800 SN_2
chr1 801 1200 SN_3
chr1 1201 1600 SN_4
chr1 1601 ...
0
votes
5answers
46 views
How to add a header to the output of an awk command afterwards?
I create a file (output) from another file (input) using awk (skipping the header):
awk 'NR==1{next} $3==1 {print $1"\t"$2}' input > output
I then have header information I can only calculate ...
-1
votes
1answer
30 views
How to find similatrites between two files? [duplicate]
Hi i am new for linux i have two files in that one have sequence_id (file_1) and another has sequence_id with sequence (file_1). I have to do if the if the sequence_id (file_1) present in file_1 to be ...
0
votes
3answers
55 views
separate one data from one column by repeated characters into as many columns as the uniq characters
So, I have a file like that:
file1: three columns
SNP Id Geno
1 a AB
2 a AB
3 a BB
. . .
. . .
. . .
1 b AB
2 b BB
3 b AB
. . .
. . .
. . .
1 c AA
2 c AB
3 c AA
. . .
. . .
. . .
and I need a file ...
-2
votes
0answers
36 views
How to convert JSON file to XML file using bash script? [closed]
Hi I need a bash script that takes a json input and converts it to xml. There is no Channel_Number record in the json input, so the script it has to be create and inserted into the xml. Below ...
0
votes
0answers
55 views
How to search and replace text using awk linux script?
I want a single CGI file that I can run on Cron that just does a simple search and replace on two small text files on my HostGator server, running Linux.
In one directory on my server, I have a text ...
-1
votes
5answers
60 views
Reading data from an input file and placing the data in appropriate fields in an xml
I have a input file and an xml file.
input file-
/user/sht
227_89,45_99
/user/sht1
230_90
/user/sht2
441_50
The file has alternate lines containing path and position.
xml file-
<aaa><...
-1
votes
3answers
57 views
Finding the different possible combinations
File A has rows of genes:
A,B,C,D,E
P,Q,R
G,D,V,K
L,Q,X,I,U,G and so on.
Taking each row at a time, how can one get the following type of output:
For the first row:
A,B,C
B,C,D
...
-1
votes
4answers
74 views
How to print <x> lines before and <y> lines after <z>th line
The question asked here asks for some lines before and afer pattern match.
But here objective is to take a line number and to fetch some lines before and after it from a file
EG:
seq 10
1
2
3
4
5
6
...
1
vote
2answers
31 views
How does `getline` work in AWK?
I've put together an example using the AWK getline function and it's confusing me.
$ cat in
foo
bar
baz
$ awk '{ getline tmp; print tmp; print $0 }' in
bar
foo
bar
baz
I'm reading the next line into ...
1
vote
4answers
70 views
Insert new line with value when not value at all is found
I am trying to find here some previous question that I can use but unfortunately can't find my exact case.
I would like to get from the output of another command that looks like this:
pattern.d
17....
-3
votes
0answers
30 views
Parse and rewrite CSS with Electron and ReactJS [migrated]
I'm writing a desktop application with Electron and ReactJS that edits CSS files. I need to scan the CSS looking for a class selector, and then clear the following declaration block and add some new ...
-2
votes
2answers
34 views
Add lines at end of csv
I have csv file with 100 rows and only one column. I need to add 100 rows of commas at the end of it. How do I do it?
2
votes
0answers
18 views
Timed display of text in terminal, perhaps slight markup (bold, colors). A.K.A. “play .srt subtitles”
Independent of the media player, I want to display separate subtitles in a scrolling text terminal to make language learning easier, or when I want more/different subtitles than the other viewers, ...
0
votes
1answer
24 views
how to delete rows with less than 2 columns in a data file in linux?
If the input look like this:
1
2 4 5 6 7 19
20
22
24 26 27
29 30 31 32 34 40 50 56 58
234 235 270 500
1234 1235 1236 1237
2300
I want to delete the rows with ...
