awk - pattern-directed scanning and processing language
0
votes
0answers
7 views
I can't print blank lines when reading values from a text file using awk in a bash script
awk is behaving very strangely in my script. I am trying to read values from a text file (see file.txt below). awk is ignoring blank spaces in my script but when I test the awk statement on its own ...
1
vote
1answer
29 views
Retrieving fasta sequences using bed file information from locally installed file
I have a .bed file containing around 30000 rows for which I have the sequences retrieved using fetch-sequences module of the rsat tool (http://rsat.ulb.ac.be/rsat/help.fetch-sequences.html#usage). ...
0
votes
1answer
16 views
Match first fields of two tab separated files and print matching values
I have two files.
Match the 1st column of first file to 1st column of second file and print entire line of the second file.
Second file is a <tab> separated file.
I have tried many awk one ...
3
votes
2answers
46 views
Find and replace different columns using awk
I try to make a change in a file:
Find a certain value and substitute it with another value in a different column using awk.
Input (UiO-66Zr-EH.mof):
unit ntype qqatom
1 'Zr1' 0.0d0
vibration
...
3
votes
1answer
140 views
Awk print problem
I have a code that takes values from a log file and populates them into an excel spreadsheet:
function ltrim(s) { sub(/^[ \t]+/, "", s); return s }
function rtrim(s) { sub(/[ \t]+$/, "", s); return s ...
1
vote
3answers
21 views
How to truncate only given column length?
INPUT:
a@notebook:~$ cat in.csv
'XYZ843141'^'ASDFSAFXYVFSHGDSDg sdGDS dsGDSgfa assfd faSDFAS saDFSAFD adFSA343fa sdfSADF'^'BAAAR'^'YYY'^'..... and so on, further columns
...
1
vote
1answer
20 views
awk getline not working as expected on first loop iteration
Consider this simple awk calculator program and its output
$ awk '{while (getline) {n=0;for(i=NF;i>0;i--)n+=$i; print "sum: " n;}}'
1 1
2 2
sum: 4
3 3
sum: 6
Note how the first input line 1 1 ...
3
votes
3answers
166 views
calculate sum of squares using shell script in perl/awk
I have 2 files as below.
file1
0.34
0.27
0.32
file2
0.15
0.21
0.15
Now, I would like to calculate the sum of squares between each column. For example,
[(0.34 - 0.15)^2 + (0.27 - 0.21)^2 + ...
-12
votes
0answers
46 views
2
votes
2answers
50 views
new pattern required from an input file using sed or awk
I have data as below:
MGW: VMG110
836-16
836-18
836-19
336-20
836-23
MGW: VMG120
3802-1
3802-2
3802-3
3456-1
3456-4
Required output is:
VMG110:836-16&-18&&-20&-23
...
1
vote
1answer
21 views
How to echo from specific line of a file to another specific line [duplicate]
Assume I have a file with about 10000 lines.
How can I print 100 lines, starting from line 1200 to line 1300?
2
votes
3answers
41 views
Get the last minute dated lines from list
If I have an input file with a list of users and their dates they were last signed in, how can I select with a sort | awk function the last minute lines?
The dates are written in a +%H:%M:%S format.
...
4
votes
1answer
49 views
Is getline in awk command safe to use?
I got weird comments on posting an awk answer on SO which uses the getline function.
Here is the link to that answer.
After posting my answer, a user came up with the below comment,(I don't criticize ...
3
votes
2answers
42 views
awk: Blank line with date
I use the following code to write the online users' process numbers and the date they were checked:
w | tail -n +3 | awk '
{ "date +%H:%M:%S" | getline tim}
{if ($1 != "")
...
5
votes
2answers
46 views
Grepping for a block of text with parts that can be optional
I have multiple entries that describes an event in a very large log file, say A.log. I would like to do two things with the event entries in the log file:
Count the number of occurrences of each ...
2
votes
3answers
42 views
replace a character from a string that is in between first and second search using sed or awk
I need to replace _ (underscore) with ? (question mark) in a file containing email addresses.
The file looks like below:
EFT_020034-E015133 20140624 /ACC/[email protected] SHR ...
3
votes
2answers
121 views
Search a string with last two characters as numeric
Suppose I have a file like this:
adasfddfd09
dsassd90897
323sdsdsdsd
sdddsdf56
dfdf45fdfdf
I want to search only those strings which have last two characters numeric with sed, grep and awk.
1
vote
4answers
38 views
Printing a new line using awk
I have a load of files in which I need to take out specific rows and then put the data I've taken out into a spreadsheet. An example would be my file shows:
Name: w
Age: x
Height: y
Weight: z
I ...
4
votes
4answers
176 views
Redirecting Output of print to multiples files in awk
I want to print same lines to 2 different files. What I am doing right now is:
print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file1.txt";
print ID[i]" "Value[i]" "TS1[i]" "TS2[i] > "file2.txt";
...
4
votes
3answers
143 views
Extracting text using sed does not work as expected [duplicate]
I am just starting out using sed and I intend to use it to extract IP addresses from ping output. Here's what I am trying to achieve exactly:
input text:
ytmti (192.188.2.3) jjggy
desired ...
0
votes
1answer
33 views
using awk for subsetting fastq file based on length of sequence
I have a fastq file. I will explain what it is. It is something like this
@SRR1024120.7 DBRHHJN1:259:D0PM7ACXX:1:1101:1386:1189 length=100
GATACAGGATGCCTGGGTCTAGGCTGTGTGACCTTGGGCCAGTTCCTCTC
...
2
votes
4answers
93 views
How to initialize an array using awk and bash?
I am trying to store values of the first line of a text file into an array. Here is what I have so far:
arr_values=()
awk '
NR==1 {
for (i=0; i<=NF; i++)
...
1
vote
2answers
84 views
Extract Values from simple html file via grep/awk
I have another grep/awk/sed issue within the attached HTML code below.
Given is a simple HTML file with table inside. The HTML is generated by a smart meter (household energy meter). Inside the html ...
0
votes
3answers
48 views
multiplication of every line
I would like to calculate a value on every line of a file such as this:
1 1441792,11534336
2 1048576,8388608
3 1441792,1153433
4 1966080,15728640
5 393216,3145728
6 2621440,20971520
7 ...
0
votes
1answer
41 views
I am having some technical issue with awk code [closed]
This is related to one of my previous question: Comparing text files using Bash and AWK
file1.txt
Name Col1 Col2 Col3
-----------------------
row1 1 4 7
row2 2 5 8 ...
7
votes
4answers
166 views
Groupwise uniq command?
I am searching for a command to get from a file in this format:
hello 32
hello 67
hi 2
ho 1212
ho 1390
ho 3000
To this format (deduplicate by taking the last row of a "group"):
hello ...
2
votes
3answers
181 views
Comparing text files using Bash and AWK
I have two text files, and I want to compare their corresponding values according to their rows and columns. By comparing, I mean to check if the values are equal and echo if the values are the same ...
0
votes
2answers
39 views
awk to Read between two Particular Strings and discard the rest
I'm using awk to just read text between two strings "Check" and "Result". I've used many variations of it found on the internet and still can't get the favored result. I've tried:
awk ...
0
votes
1answer
25 views
Creating a file from another file on the basis of a numeric value in each line
We will generate a csv file with below values.
yp1234,577,1,3
yp5678,570,3,5
yp9012,132,8,9
I need to extract data and create files based on second column. If it's value greater than 577 then the ...
3
votes
3answers
141 views
Replace multiple strings in a single pass
I'm looking for a way to replace placeholder strings in a template file with concrete values, with common Unix tools (bash, sed, awk, maybe perl). It is important that the replacement is done in a ...
1
vote
1answer
42 views
Awk that I am stuck on
I have a project I am working on for work. The script I am building initially takes two arguments: one is an unique Identifier and the other is for a log file (both strings). Now I have the first part ...
2
votes
3answers
61 views
Use awk results as parameters to another command
I'm extracting rows from a set of text files with awk. The files look like this:
1000 1 75
1000 2 76
1001 1 76
1001 2 80
I'm searching several directories of these with this ...
3
votes
3answers
129 views
awk filtering live capture with file as pattern list
What I'm trying to do is quite simple. I'm generating output from tshark and redirecting it to awk with a pipe |. Since tshark gets live data, I want that awk search in every output for a pattern ...
6
votes
8answers
535 views
Grep rest of line…after match
I have a file containing only two lines, with the following structure:
$ cat /tmp/pwpower.log
000D6F0000D34227, -114.10
000D6F0001A405C4, -130.09
The values are power values of my solar plant. ...
3
votes
3answers
122 views
AWK: Passing shell variables to awk
I am trying to pass a variable number of arguments from the shell script to a pattern recognition subset of a table. Here is my attempt so far:
The file 'infile':
ID,GROUP
1,GROUP2
...
4
votes
3answers
125 views
UNIX paste columns and insert zero's for all missing values
I would like to merge specific columns from two txt files containing varying number of rows, but same number of columns (as shown below):
file1:
xyz desc1 12
uvw desc2 55
pqr desc3 ...
1
vote
1answer
26 views
Move data row(s) to single column while retaining row header(s)
I have reports that are generated in the following tab-delimited format:
UNIT TC CC PC TCP FTX FRX
HOUSE 55 65 75 85 95 105
CAR 100 200 300 400 500 600
H2 ...
1
vote
3answers
90 views
awk for loop for each line in a file
I'm a newbie to bash scripting. I need help trying to figure out how to get loop through a text file and for every line take the fields of name , project #, and email, and replace them in a email ...
2
votes
2answers
52 views
How to print top five highest numbers from a column?
I have a text file with four columns. I need to read till end of file and print the highest number from column3 (top 5 values) along with column 1.
input.txt
xm|340034177|ref|RT_235820.1| 139697 ...
3
votes
2answers
82 views
How can I derive an IPv6 address from a MAC address?
We know that we can use the MAC address to create an interface identifier, e.g. for a link-local IPv6 address which should be unique in the Network.
The image shows the way to do this:
My ...
-1
votes
3answers
32 views
AWK: Nested Conditional Subset of Rows without Headers [duplicate]
I have a pretty specific question, and have been able to find lots on conditional subsetting using awk but none that lends enough explicit code for me to generalize to my situation. I have a file ...
1
vote
2answers
70 views
printing special characters in awk output
I have a large file that contains lot MQ messages with RFH2 header. Each message in the file is separated by a blank line. Now I need to split this large file into small files each containing single ...
1
vote
1answer
48 views
AWK: Nested Conditional Subset of Rows
I have a pretty specific question, and have been able to find lots on conditional subsetting using awk but none that lends enough explicit code for me to generalize to my situation. I have a file ...
1
vote
5answers
82 views
Use SED or AWK to move string to new column and row
I have a .csv file that is tab delimited and I need to move the header row information to a "new" column at the beginning and a specified numbers of rows down. Please forgive any formatting errors for ...
2
votes
3answers
118 views
How to replace the content of specific column with awk?
Given there is 40 columns in a record. I want to replace 35th column so that the 35th column will be replace with the content of 35th column and a "$" symbol. What came up my mind is like:
awk ...
1
vote
3answers
93 views
GREP / SED or AWK: Print entire paragraph in a file on pattern match
I have a file with hundreds of paragraphs of around 15 lines each. I need to search for a pattern, say Occurance: 1. If this pattern is found in the para, I need to print the entire paragraph. Note ...
0
votes
1answer
47 views
Search a string between two timestamps starting from bottom of file
I was trying to find a string Cannot proceed: the cube has no data in huge test.txt file only between timestamps yesterday 22:30 pm to today 00:30 am.
Script:
tac test.txt | awk -v today=$(date ...
1
vote
2answers
62 views
Compare set of values from two files and print the output in third file in awk
Hi friends i have problem with comparing two files.I'll try my best to explain.I have two file f1 and f2 which i have to compare on basis of two sets Parameter(p) and Value(v).
Basically f1 contains ...
1
vote
3answers
31 views
Shell command to remove the first string that is enclosed with slashes
I need a shell command(sed or awk is preferred) to remove the string that is enclosed within the slashes.
For example If my string is /some/dir/example/ then the output should be dir/example/
4
votes
3answers
309 views
awk command for parsing a file
I have a following text file. I am showing you the first 3 lines.
chrom st end gene strand c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13 c14
chr6 3345 3543 geneA + 36 -23 ...
