Manipulation of text by programs, scripts, etc.
7
votes
8answers
106 views
Split text file into lines with fixed number of words
Related, but no satisfactory answers: How can I split a large text file into chunks of 500 words or so?
I'm trying to take a text file (http://mattmahoney.net/dc/text8.zip) with > 10^7 words all in ...
1
vote
2answers
33 views
With awk, how to Insert a number only for lines starting with “Toto” when they are between a specific couple of patterns
With awk, I would like to insert a numbering like Record n°i# in front of each line starting with Toto only when they are located in-between two specific patterns start=ABCD and stop=EFGH.
Input file ...
2
votes
3answers
41 views
How to add “wrappers” around methods in source code based on a pattern, using sed, awk, grep and friends
How can I add wrappers to a file based on a pattern?
For instance I have the following:
...
find(css_policy_form_stage3).click
with_ajax_wait
expect(css_coverage_type_everquote).to ...
0
votes
1answer
23 views
How to return sequence value in sqlldr from unix shell
I have a test.txt file with column (Serialnumber,value).
test.txt:
"SEQ_IN_AMT.NEXTVAL",123123
"SEQ_IN_AMT.NEXTVAL",123114
here SEQ_IN_AMT is a sequence name create on Oracle. Now, when I load ...
0
votes
2answers
19 views
Replace values on the basis of matching ID variable
I have two files. file1 has 6 columns, file2 has 2 columns. I want to replace the values in the 6th column of file1 by the values in column 2 of file2, based on matching ID variables (column 1 in both ...
1
vote
3answers
42 views
Replacing a “time” column with corresponding values
I have a file in the following format:
"2004-04-19 12:25:57" 44 44
"2004-04-19 13:39:32" 36 36
"2004-04-19 14:00:53" 34 34
I need 2 new files:
a) A file which will replace "time" values of the ...
5
votes
8answers
340 views
awk or sed command to match regex at specific line, exit true if success, false otherwise
I need to determine if a file contains a certain regex at a certain line and to return true (exit 0) if found, and otherwise false. Maybe I'm overthinking this, but my attempts proved a tad unwieldy. ...
2
votes
4answers
71 views
Cost-efficiently pair each line of a file with all others
I have a very huge file containing numbers only.
file -
123212
234234
12324
1243223
5453443
And want to pair each line with all others. Output like below
123212,234234
123212,12324
123212,1243223
...
2
votes
2answers
57 views
How can I compare two files and create another file that aren't in first file?
file1 contains:
100
200
300
400
file2 contains:
500
600
700
200
300
Expected output file will be:
500
600
700
-1
votes
3answers
37 views
How to delete certain line from file?
I have a text file with line containing only comma. How can I delete this particular line? Consider following example:
CREATE TABLE ENTERPRISE.TSDAILYACCTSUMMARY (
TSDAILYACCTID FOR COLUMN ...
-1
votes
1answer
48 views
How to add double quotes to a various strings in a same pattern in a file
I have a file like this
CREATE TABLE ENTDTA.$$BAPWD (
PERSID NUMBER(10) DEFAULT 0 NOT NULL ,
"PASSWORD" CHAR(10) DEFAULT '' NOT NULL ,
UPDATE_IDENT NUMBER(7, 0) DEFAULT 0 NOT NULL )
...
2
votes
1answer
28 views
Merge text files with 1st column and difference in output file
File 1: (text1.txt)
Row_Added_Ts Count
01/01/14 2022448
02/01/14 8345155
03/01/14 8663852
04/01/14 6785739
05/01/14 5279913
File 2: ...
1
vote
1answer
36 views
Merge lines with matching first column
I've been trying to use awk to merge two files when they have the same first column. Here are my example files:
FileA.txt
A2M 1
A4GALT 11
AAAS 35
AAGAB 7
FileB.txt
A4GALT 2
AAAS ...
-1
votes
1answer
22 views
Look for a particular pattern and do a swap on that line
I have a set of files containing
VIEW_NAME FOR COLUMN DNAME ,
VIEW_OWNER FOR COLUMN DCREATOR ,
OBJECT_NAME FOR COLUMN ONAME ,
OBJECT_SCHEMA FOR COLUMN OSCHEMA
So, ...
0
votes
1answer
21 views
replacing first link of a text “awk equivalent of sed command”
I wanted to replace the first line of a text with sed it should be like
sed -i.bak "1 s/^.*$/"CompoundState\tMethod\tApproach\tS^2\tEnergy\tPath"/" awk.xls
but I also need to run it on OSX but it ...
4
votes
2answers
221 views
How to change all strings in python file from snake_case to camelCase in sed
I've tried some sed patterns like this from commandlinefu
sed -r "s/('[a-z]+)_([a-z])([a-z]+)/\1\U\2\L\3/"
But somehow it's not working. For one thing they forgot the digits, which I can fix, but ...
1
vote
4answers
94 views
Show ratio for all availiable resolutions
I tried to get the ratio with this script:
echo "$(xrandr)"|cut -f 4 -d" "|sed 's/x/00 \\\* /g'|while read i; do echo "$i = "$(expr $i); done
but I get errors like
102400 * 768 =
expr: ...
7
votes
4answers
210 views
How to grep -v and also exclude the next line after the match?
How to filter out 2 lines for each line matching the grep regex?
this is my minimal test:
SomeTestAAAA
EndTest
SomeTestABCD
EndTest
SomeTestDEFG
EndTest
SomeTestAABC
EndTest
SomeTestACDF
EndTest
...
3
votes
2answers
82 views
Using grep to print all of the matching strings not separated by a space
I'm looking through a text file of emails and trying to get a list of ones from a certain domain.
I'm running:
grep -oh "\w*domain.com\w*" file.txt
This finds the lines, but only prints that ...
6
votes
4answers
198 views
How to add a header and/or footer to a sed or awk stream?
I have a bunch of output going through sed and awk.
How can I prefix the output with START and suffix the answer with END?
For example, if I have
All this code
on all these lines
and all these
...
4
votes
1answer
56 views
How to use awk to indent a source file based on simple rules?
How can I indent source code based on a couple of simple rules?
As an example, I've used sed and ask to transform a selenium HTML source table to the following rspec like code. How could I ...
2
votes
1answer
32 views
Filtering and extracting PIDs as space-separated list from ps using sed
I want to filter and format the output of ps -ax -o pid,command, but not sure what pipe to use for it.
I want to filter those lines matching a fixed string at a fixed position, just keep the pids for ...
0
votes
5answers
40 views
update the json file
I have a file, actually json file. How I can add a comma to the end of every but the last line ? For example:
{"a":1, "b":1, "c":2}
{"a":3, "b":3, "c":1}
{"a":1, "b":2, "c":3}
What I want:
{"a":1, ...
2
votes
2answers
48 views
adding column in CSV file using awk
I need to a add a column in CSV file from an array using awk.
For example,
input.csv
a,10
b,11
array = (100 200)
output.csv should look like
a,10,100
b,11,200
I tried this command but it does ...
2
votes
1answer
37 views
Question about replacing character in text file
So I have some data that is parsed in this format seperated by colons:
username:[email protected]:password:salt
Sometimes the username/salt can contain a colon (:) in it.
I want to replace the ...
2
votes
3answers
93 views
FNR and NR variables problem
I have two files, one of them has 10 records, and the other one with 15 record, the problem is when I cat them together, always FNR=NR.
For example, consider below files:
File1:
1,boo
2,foo
3,boo
...
0
votes
2answers
38 views
How to search between the 2nd and 3rd delimiters
So, I have a data file and I would like a match to occur only if a match is found between the 2nd and third Vertical Bars (|)
So given this data sample, I if I search for 'wilson' I want the 2nd line ...
3
votes
4answers
220 views
Conditional if clause in awk
Please consider below file:
foo,1000,boo,A
foo,1000,boo,B
foo,1001,boo,B
foo,1002,boo,D
And we have below rules:
If $2 equal 1000, $4 should be equal A
If $2 equal 1001, $4 should be equal B
If ...
0
votes
1answer
37 views
How to add new column, where the value is based on existing columns with awk
I have a file of multiple columns. I would like to make an additional column based on the values of 2 columns from this file.
Example input:
A B C D E F
1 2 T TACA A 3 2
3 4 I R 8 2
9 3 A C 9 3
...
3
votes
3answers
48 views
Find a pattern and insert # at the beginning of 2 line befoe that and 1 line after that
I have a big file (more than 2000 lines). Where i have to insert a # at the beginning of 2 above lines and at the beginning of 1 below line after finding a pattern. Also insert # at the beginning of ...
1
vote
1answer
49 views
How to print the line numbers with corresponding line that matches a pattern using shell command? [duplicate]
File,
.false alarm is bla no. 11
.no alarm generated is bla
.application will be killed is bla wall command
.doc file is document file with .doc extension
.no authority for this selection bla
Now ...
1
vote
2answers
32 views
How to replace string in files except with a line begins with # [duplicate]
I've looked at some other questions related to using sed to replace strings, but I'm having trouble figuring out how to apply it to my particular need.
In a case like the snippet below, I want to ...
4
votes
6answers
201 views
Changing matrices into single lines of data
I have multiple matrices in a .txt file and I need each matrix to be in a single line. For example,
matrices.txt:
1 2 3 4
2 3 4 5
3 4 5 6
3 4 5 6
2 3 2 5
2 3 4 5
2 3 5 6
2 3 4 5
...
What I would ...
1
vote
5answers
120 views
format file to remove " characters
I have a file with the following data
"MG1507XXXXXX|" "|020000XXXXXX" "20261031|" "|3,827.92" "|3,581.41" "|542,729.62" "MBA"
"MG1507XXXXXX|" "|020000XXXXXX" "20261130|" ...
0
votes
5answers
68 views
Appending One File to the Second Until End of File
I have two files. The first is just a single number. This number is calculated and changes every time. The second file has a bunch of columns, and I do not know the number of rows (which can change). ...
1
vote
3answers
24 views
fetch a column from a file based on another column
I have a file which contains approximately 5 million records, as follows: -
1223423,21,foo,data1,data2,data3,data4,data5,45,267,index1
4234234,34,bar,cat1,cat2,cat3,cat4,cat5,34,2323,index2
...
0
votes
1answer
82 views
How to replace \1\ with new line
I have a record like:
"evSchema"\1\"UAT" "evSN"\1\"uadb" "evDirsep"\1\"/" "evRootPath"\1\"/work_area/APP_UAT/" "evSchema"\1\"RMS13" "evUser"\1\"STAGE"
I want to replace \1\ with new line.
I tried ...
0
votes
1answer
33 views
Repeatedly extract every second and third line of a file with 3-line data blocks
I have a file where I have data in 3-line sets. I want the 2nd and 3rd line from each set.
How do I get that?
For an example from: [set no][no of line in set]
11
12
13
21
22
23
31
32
33
I want:
...
4
votes
4answers
134 views
Current date in awk
Is there a constant variable in awk, that store today's date? if no, is there a way that can store today's date for daily use?
let's say we have below file:
boo,foo,2016-08-30
foo,boo,2016-07-31
...
12
votes
4answers
371 views
Sort lines by number of words per line
Given input:
hello: world foo bar baz
bar:
baz: bin boop bop fiz bang beep
bap: bim bam bop
boatkeeper: poughkeepsie
I would like to sort it into most words at the top, to least at the end, like ...
2
votes
3answers
65 views
remove lines that contain an specific text in a file
I have a file with a list of emails in it and each line has an email in it. I want to remove lines that contain example.com or test.com.
I tried this:
sed -i 's/example\.com//ig' file.txt
But it ...
2
votes
2answers
67 views
Extract multiple instances of text between two words inclusive of starting word but exclusive of ending word
A PDB file contains numerous paragraphs of conformations of a protein.
Each conformation starts with the keyword ATOM and ends with keyword END.
I am trying to read the file in bash such that I ...
0
votes
1answer
42 views
Read all the values of a particular column of a file and store it in another file
I have a file.txt which contains data like below:
col11 col12 col13
col21 col22 col23
I fetch 2nd column of all the available rows (no. of rows may vary) using awk '{print $2}' file.txt
Now i ...
0
votes
3answers
49 views
Append a string to the beginning of second line using unix
I want to append a string to the beginning of second line of my text file, for example:
1
2
3
should become:
1
42
3
Any idea about how to go about with this thing?
0
votes
2answers
42 views
Monitor a logfile and execute different commands depending on different conditions
I have a logfile which I want to monitor and depending on which condition is met, different commands should be executed.
I found a solution that comes close to this here. Unfortunately it doesn't ...
5
votes
5answers
227 views
Merging columns in a file using Awk
Input:
ABC,SSSD,12345,NSS,12345,xxx,TS11
,,,,,,TS21
,,,,,,TS22
,,,,,,BS26
,,,,,,GPRS
ABC,SSSD,12356,NSS,12356,xxx,TS11
,,,,,,TS21
,,,,,,TS22
,,,,,,GPRS
,,,,,,BS26
Output:
...
1
vote
3answers
54 views
Find lines in (muliple) files that contain the given line in an input file
So, if I have a number of files like the following:
file1:
123
456
789
012
file2:
line1 922
line2 392
line3 456
line5 291
line6 201
...
file3:
line1 111
line2 123
line3 19
line5 542
...
1
vote
2answers
49 views
How to subtract lines, in reverse, only if greater than previous line
I am working on a shell script that will read the lines of a file, in reverse. I need it to subtract the previous line from the latter, but only if the latter is larger. If it's smaller, then I just ...
0
votes
2answers
32 views
How to add a new text after every matched string using shell command?
Input:
Job name: ns, Job ID: 2312, Status: ODB_ACTIAVTION SUCCESSFUL
Job name: ps, Job ID: 3353, Status: ODB_ACTIAVTION SUCCESSFUL
Job name: pm, Job ID: 1265, Status: ODB_ACTIAVTION SUCCESSFUL
Job ...
2
votes
4answers
58 views
Join pattern line and consecutive non-empty lines, separated by commas and enclosed in ()
I have a file that has
click
css_add_violation_false
click
css_add_claim_false
and
select
css_driver1_birth_date_month
birth_date_month
select
css_driver1_birth_date_day
birth_date_day
How ...