awk - pattern-directed scanning and processing language
0
votes
2answers
91 views
Extract data in linux/unix
I have input data which I want to parse and extract values using awk/grep/sed:
group-2 9 10 text color=black,from=ABCB11,fromid=4,order=2,thickness=3,to=ACE,toid=11,use=1,z=1
group-3 0 1 text ...
-1
votes
0answers
34 views
Write a script file 'account.awk' that from a file 'orders' creates the formated printout of account
Write a script file account.awk that from a file orders creates the formatted printout of account. The file orders is as follows:
http://pastebin.com/kPF5B0c0
Beside the column Cost the account ...
2
votes
1answer
124 views
Regex doesn't work with grep
I have a string like this
TEST="/dev/sda1 on /media/sda1 type vfat (rw,relatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,quiet)"
(Result from mount | grep sda1)
...
1
vote
2answers
24 views
How to multiply two columns in awk?
I want to multiply column 1 with column 2 (till end of file) in input file and should output 1 column and multiplied 3 column in separate file.
input.txt
1 677679866
2 121867616
3 49413198
4 ...
0
votes
2answers
40 views
Compare first column in different files
I want to compare one file with other files in the same directory.
file1.txt contains:
move 34#123#
get 11#278
put 33#29#567#23
file1.txt should compare with files 1.txt and 2.txt.
1.txt ...
0
votes
1answer
34 views
How can I print a specific occurence of a range pattern?
I am trying to extract several data fields out of somewhat peculiarly formatted strings.
Each string has a format similar to the following (spaces added for emphasis):
1 abcd 2 1 efg 2 hij 3 ...
7
votes
5answers
237 views
SED : Case Match Pattern Replacement
I have a source code spread across several files.
It has a pattern abcdef which I need to replace with pqrstuvxyz.
The pattern could be Abcdef (Sentence Case) then it needs to be replaced with ...
3
votes
2answers
126 views
How can I print out all but the last n characters in bash?
I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command.
All I can find through searching is using sed, ...
3
votes
2answers
42 views
Printing in awk up to specific line and 5 more results
I'm trying to print out all the lines up to the appearance of a particular song and 5 more songs with that. The problem i'm new to the shell what I have to fix in the bellow commands?
RANK NAME ...
2
votes
1answer
34 views
I'm trying to count and print song file names using awk as following
RANK NAME BAND YEAR GENERE DOMESTIC/INTERNATIONAL
206:Reach Out, I'll Be There:The Four Tops:1978:Pop:3/2
207:Bye Bye Love:The Everly Brothers:1950:Classic:3/2
...
-1
votes
0answers
20 views
Rearranging text file with blocks of data [duplicate]
I have a file whose content looks like this:
2009 150 0 0 0 0000
75.316 0.0400390625 0.00007 0.00000 0.8980
76.216 0.0400390625 0.00007 1.00000 0.9046
77.217 ...
3
votes
4answers
140 views
Rearrange text file by prepending data rows with header row
I have a file whose content looks like this:
2009 150 0 0 0 0000
75.316 0.0400390625 0.00007 0.00000 0.8980
76.216 0.0400390625 0.00007 1.00000 0.9046
77.217 0.0400390625 ...
2
votes
3answers
59 views
Append columns in a text file to after the final row
A smaller version of many data files I have looks like this:
0 0 0
0.05 9.6877884e-06 0.0024898597
0.1 4.2838688e-05 0.0049595502
0.15 0.00016929444 0.0074092494
0.2 0.00036426881 0.009839138
0.25 ...
0
votes
1answer
46 views
How to assign variable in awk
I have a code like below:
$ awk -F'[]]|[[]' \
'$0 ~ /^\[/ && $2 >= "2014-04-07 23:00" { p=1 }
$0 ~ /^\[/ && $2 >= "2014-04-08 02:00:01" { p=0 }
...
1
vote
3answers
51 views
Print the number of values which have the special characters in them
I have a text file like the one below. I want to check the second column and print the count of values which have some special characters in them.
101,aaa,d01
102,*&%,d02
103,$%&,d03
...
3
votes
2answers
44 views
Merging multiple lines based on column 1
I have a file like below..
abc, 12345
def, text and nos
ghi, something else
jkl, words and numbers
abc, 56345
def, text and nos
ghi, something else
jkl, words and numbers
abc, 15475
def, ...
3
votes
4answers
64 views
merge 2 rows based on the same column values
I have a file like below.
47196436 47723284 name1 1.77273
42672249 52856963 name2 1.06061
52856963 430695 name2 1.16667
55094959 380983 name3 1.55613
17926380 55584836 name4 1.02461
3213456 34211 ...
3
votes
3answers
58 views
awk - to add a column after matching the remaining columns
I have a file (file1.txt) with the contents as below.
8145216 3377090 1.5
1405541 53595498 1.53637
I need to generate all the possible combinations for the first 2 columns in the above file. I use ...
1
vote
1answer
33 views
linux if else fails for some IP's
i am using the following code with if then to mount. It works for some IP's, it does not work for some IP's especially for FAILURE statement.
Here is the code
is_mounted() {
mount | grep -qohw ...
7
votes
5answers
323 views
How to extract logs between two time stamps
I want to extract all logs between two timestamps. Some lines may not have the timestamp, but I want those lines also. In short, I want every line that falls under two time stamps. My log structure ...
2
votes
2answers
107 views
How to do alphanumeric validation for a particular column (say $2) of a file using awk?
How do I perform alphanumeric validation for a particular column (say $2) of a file using awk?
For example I've tried:
awk -F, '{if ($10==(/[0-9a-zA-Z]/)) count+=1;} END {print count}' final.csv
I ...
0
votes
2answers
58 views
Extracting with grep/sed a first pattern and then a second pattern that occured some lines before and related to the first
I'm trying to extract a specific line from a .htm file when a pattern occur ("Event 100" in my example below) but at the same time I need to retrieve also another pattern which is related to the first ...
1
vote
1answer
63 views
AWK - every thing in one line? [closed]
Hello can I make several changes in a file using just on line with awk?
For example, transform this:
awk < towhee_input5 -v k="$nstep" 'NR==18 {$0=k}{print}'> towhee_input6
awk < ...
2
votes
3answers
107 views
Merge 2 files based on all values of the first column of the first file
I need to merge below 2 files:
file1:
TABLES REF-IO HEAD-IO DIFF-IO
test 200 500 -300
exam 2 3 -1
final 2 1 1
mail 4 2 2
TOTAL 208 506 -298
...
1
vote
2answers
83 views
How to replace a custom date string in a text file
I've looked through several tutorials and a couple of StackOverflow posts, but I've not seen any examples using sed or awk which use dates. I have a text file that contains this as the first line, I ...
4
votes
7answers
484 views
How to split a filename list in 5GB sets?
I have a pretty big list of files which I like to group in smaller list of 5GB each for further processing.
The file list is in the following form: "size" (from du) "tab" "absolute path", like this ...
3
votes
4answers
204 views
extract the characters before () using grep
I need to extract the characters before the character '('. My file structure is:
rose(good)ook
daisy()no
lilly(bad)fine
sunflower()nice
I need output file as in the form of:
rose
...
1
vote
3answers
53 views
awk want to total by hour and variable
I am able to run this and it works fine - note: field $1 is a time/date stamped field..
gawk -F ":" "{ print $1 }" /cygdrive/c/counting/ourlog | sort | uniq -c | sort -r
57339 2014-03-21 09
54290 ...
1
vote
1answer
52 views
Remove the string before the whitespace
I want to remove a word before the whitespace. My file structure is:
ant
tiger
static rabbit
lion
dynamic monkey
donkey
I need the output as:
ant
tiger
rabbit
lion
monkey
donkey
...
3
votes
2answers
58 views
introduce line break at the same position based on another file
I have 3 input files all having the same number of lines. I have come up with a movie example to make the question look interesting.
file1.txt -- Essentially contains the actor names.
...
1
vote
2answers
75 views
Union of two files and unique in new file
I have two files with few similar column one, two and third column with small variation and few other columns as follows.
file 1:
AT1 AT22 24 1 ATAGATA ATTATAT
AT2 AT24 22 0 ATAGATA ATTATAT
...
5
votes
4answers
467 views
Reduce the over use of files in text processing
Currently, I have a file (Results.txt) which has 27k rows. The file content is as below.
45008657 | A. | | Long | 0
49210987 | A. | ...
3
votes
1answer
28 views
update line based on previous line
I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style:
12:33:41 unix,restarts
12:35:00,lofi4096,0,0.0,0,0,0.0,0.0
...
1
vote
3answers
61 views
Extract part of string using grep
I have 3 files in a directory
MYO144064T
MYO144064TA
MYO144064TX
Where the digits and 11th character will change.
In my csh script, I want to extract the letters 'A' and 'X' probably using grep
...
3
votes
3answers
372 views
How to parse JSON with shell scripting in Linux?
I have a JSON output from which I need to extract a few parameters in Linux.
This is the JSON output:
{
"OwnerId": "121456789127",
"ReservationId": "r-48465168",
"Groups": ...
3
votes
4answers
135 views
Split the input file with line breaks
I am trying to insert line breaks on a file based on the same names which happens to be the second field in my file. So, my input file is like below.
17412193|name1|organization
...
2
votes
2answers
150 views
How to find and replace a field column value in UNIX
I have file with the value say which is delimited with |
123-|aaa|bbb|123|123.0|123-|123.01-|-123.02|123.03-|aaa|bbb|123-|aaa-|-bbb|cc-cc|123.04-|aa123-|123.05-
I need to pick the column values ...
3
votes
4answers
137 views
Identifying genes from a list of genes
I have a gene list file. Some thing like this
SWT21
SSA1
NRP1
EFB1
TFC3
MDM10
I have another file which also contains the names of these genes in my list along with other ...
-2
votes
1answer
59 views
concatenate or remove multiple files of a directory
In general for file concatenation with the same extension, we use cat *.txt. But in case we are in a different directory and trying to concatenate another directory's files how to do that ?
I have ...
1
vote
2answers
69 views
gawk: Summing floats formatted with comma separator
I'm trying to sum floats which use the comma as separator, and gawk does integer calculations. Here's the script:
BEGIN {
FS=";"
OFS=";"
CONVFMT = "%2.2f"
}
{
print $1
print $2
print ...
4
votes
2answers
228 views
Merge fields in a file
I have a file with 7 columns, a GFF file having chromosomal regions.I want to collapse the rows where REGION ="exon" to only one row in the file.The row has to be collapsed on the basis of regions ...
4
votes
4answers
173 views
Pass shell variable as a /pattern/ to awk
Having the following in one of my shell functions:
function _process () {
awk -v l="$line" '
BEGIN {p=0}
/'"$1"'/ {p=1}
END{ if(p) print l >> "outfile.txt" }
'
}
, so when called as ...
2
votes
5answers
97 views
Print text between (and including) two delimiters on a line
What's the best way to print the text between (and including) the first pattern ("SELECT" or "SSELECT") and a second pattern ("[") using a command-line took (e.g. cut, sed, awk)?
For example, given ...
1
vote
4answers
101 views
How can I delete the first dot before the first number?
How to remove the first dot before the first number?
For example
I have the following file
more file
Version …………2.465.76.8.332
I want to get only the version number - 2.465768.332
7
votes
4answers
158 views
BEGIN and END with the awk command
According to the awk manual, BEGIN and END are not used to match input, but rather to provide start-up and clean-up information to the awk script. Here is the example given:
ls -l | \
awk 'BEGIN { ...
2
votes
5answers
92 views
identifying words in file 1 and matching them to second file and printing them
I have two files as follows
File 1 has list of Ids
ram
Tarah
taikah
jhdfklj
File 2 has
id=ram;*between*hkjhdswklf kjsdklhglshglkhgklsdgkhdfg
id=taikah;jhdjfkhg**kjdfkjksfdjk ...
3
votes
5answers
119 views
Merging Two Files columns in order
I want to merge two files.
Files A.txt
001;abc;def;ghi;jkl;pqr
002;abc;def;ghi;jkl;pqr
003;abc;def;ghi;jkl;pqr
004;abc;def;ghi;jkl;pqr
005;abc;def;ghi;jkl;pqr
.
Second File B.txt
001;mno
002;mno
...
0
votes
1answer
37 views
How can I use gsub with a pattern? And how to tell it which file to awk?
I've read here that I can replace the values of all columns containing a particular text using
awk -F'\t' -vOFS='\t' '{ gsub("CC", "C", $1) ; gsub("AA", "A", $1) ; print }'
But where do I tell awk ...
3
votes
3answers
196 views
awk remove the rows based on another column value
I have a file like below which I have sorted based on the username field.
UserID score UserName
1234 200 Jack
5678 150 Jill
8543 200 Jill
5678 100 John
I am trying to remove ...
1
vote
3answers
125 views
Print the last-but-one field
How to print the word before the last word in line (with ksh or awk or sed or perl one liner)
Example 1:
echo one two three
will print "two"
Example 2:
echo 1 2 3 4 5 6
will print "5"
