awk - pattern-directed scanning and processing language
0
votes
0answers
14 views
match ids and print original file
I have two files
Original: ( 5000 entries)
Chr Position
chr1 879108
chr1 881918
chr1 896874
...
and a file with allele freq ( 2000 entries)
Chr Position MAF
chr1 881918 0.007
...
0
votes
1answer
33 views
Search low values in linux files
I am working with mainframe files and it seems that the low values (x'00') exists in the records in the file which throughs off my import for fixed block
Does anyone know how to eliminate this ...
2
votes
2answers
57 views
Extract Data between comma
I have an output file which contains different numbers seperated by commas. But the issue is I do not know the number of commas.
I would in turn like this data to be grep on another file. For ...
2
votes
2answers
81 views
changing pattern by removing spaces?
file :
hi hello,new york, , ,brazil site ,brazil
january,month is feb , , ,indiana jones
task:
delete all horizontal space (spaces and tab) between any two comas (like , ,) if ...
1
vote
1answer
52 views
AWK how to add all columns beginning with specific row
I have this output:
Count Process
238 0
1 1124027
1 1124065
1 1124066
1 1124069
1 1124072
1 1124073
1 1547365
1 1547366
I wanted to add all Count ...
2
votes
1answer
42 views
shell variable in awk is not being passed to all the lines but just for the very first line of input?
inputfile:is2.txt
10.39.5.41,A1,B1
10.39.5.41,A2,B2
10.39.5.41,A3,B3
10.39.5.41,A4,B4
10.39.5.41,A5,B5
10.39.5.41,A6,B6
script :
#!/bin/bash
second_column="OOOOOOO" # OOOOOOO will be added to ...
1
vote
0answers
39 views
reading from a file and changing its pattern into an array?
I have a CSV file containging entries like this :
ipaddress,VLAN,VLANid
10.192.168.1,vlan-xyz,3
10.192.168.1,vlan-abc,8
10.192.168.1,vlan-mnp,11
10.192.163.24,vlan-llz,3
10.192.163.24,vlan-bbz,5
...
1
vote
4answers
62 views
How to find what device a file is on (and use that in a script)?
I want to find out what device my file is on so that I can use it in a script. I can get this far:
$ df .
Filesystem 512-blocks Used Available Capacity Mounted on
/dev/disk0s2 498438976 ...
1
vote
2answers
59 views
sed pattern matching with deletion over a line
I want to delete only some spaces(say 4) after the line$#*425 with space
I am not able to complete it please suggest
echo "line$#*425 with space " | sed 's/ /\1 \2/g'
i am thinking of ...
1
vote
2answers
45 views
Print fields from file only if line starts with a specific word
I have the following file:
…
LINK=dummy 172.17.100 => 10.218.11 [PATH=/etc]
FILE=hosts 172.17.100 => 10.218.11 [PATH=/etc]
FILE=network 172.17.100 => 10.218.11 ...
1
vote
1answer
26 views
Regex to Cisco Interface
I would like to know if have some Sed/Grep or Awk regex to parse Cisco interface section and exclude if interface is down (shutdown), like example bellow.
Example:
!
interface FastEthernet0/1
ip ...
1
vote
0answers
69 views
Processing shell script options with awk
I'm looking for way to process shell script arguments that is cleaner and more "self documenting" than getopt/getopts.
It would need to provide...
Full support of long options with or without a ...
1
vote
0answers
41 views
How to awk printf additional string with backslash
I have a csv that I want to add a preceding column to and a trailing column. The trailing column is a date (manual string right now). Every time I try to get that date, it ends up replacing some text ...
1
vote
1answer
86 views
How to calculate average requests per seconds using awk or sed on Apache logs
I am trying to calculate the average requests per seconds for a specific period of time of my Apache logs.
I have been able to narrow the results down to the period I am interested in, as well as ...
1
vote
1answer
95 views
Split log file by time range
I have a log file and would like to cut it into copies of 15min. In the log file the lines start with:
dd mmm yyyy hh:mm:ss,xxx
e.g.
12 Feb 2013 16:05:02,xxx log text...
and the file rotates ...
3
votes
4answers
141 views
Text between two tags
I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc.
Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
3
votes
2answers
111 views
AWK for data smoothing
I need some help to smooth some data with awk olny of column 4 and next.
This is a data example:
Date;time;Time_ms;A;B;C;D
23.11.2012;15:03:00;41236627696;1;2;2;3
...
2
votes
2answers
83 views
System calls, AWK, and Bringing in external inputs
awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty
In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this?
What I'm aiming to ...
3
votes
2answers
74 views
AWK : Converting Row Formated Data into Columnar
I have a dataset as such:
Name: Jim Bean
Vice: Dice
ID: AFDSDFDSFDSFASFA
LoginTime: 12343314
Name: Bob Dylon
Vice: Trumpets
ID: AFD232SFDSFASFA
LoginTime: 12343314
...
5
votes
2answers
159 views
what is the meaning of 1 at the end of awk script
I was reading this awk script
awk -F"=" '{OFS="=";gsub(",",";",$2)}1'
I want to know what is the function of 1 at the end of it
2
votes
1answer
59 views
How can I use awk to (sometimes) remove the first character of a column?
I run a command and pipe it through awk as below :
mycommand | awk '{print $1,tolower($3),$4}
This gives me the following output:
fred.o: t .abcdefg
fred.o: u .rstuvwxy
fred.o: t .defghi
...
0
votes
0answers
57 views
Co Record Verification
I have a task requiring some awk to verify a few records quickly.
Let' say:
A1,A2
B1,B2
C1,C2
C2,C1
A1,C1
A1,B1
B1,A1
Which would be the best way to check for reciprocity only between the A# the ...
2
votes
1answer
76 views
Parse csv using awk [duplicate]
Possible Duplicate:
Is there a robust command line tool for processing csv files?
I am trying to parse the second line in a file (which contains commas within double quotes)
"100 Watt ...
2
votes
2answers
22 views
sort a file based on length of the column/row
I need to sort a file based on the number of chars in the first column.
I have no idea on how to go about this. (On Linux, so sed/awk/sort is available).
An example:
.abs is bla bla 12
.abc is bla ...
1
vote
0answers
76 views
Conky - string formatting
I have conky the record:
${color lightblue} Down: ${downspeed wlan0} Up: ${upspeed wlan0}
Conky prints link speed, but the text moves:
Down: OB Up: OB
Down: 60B Up: 60B
Down: 148B Up: 148B
...
3
votes
3answers
93 views
How can awk cut certain fields and add to the end of each line?
I would like to convert a list containing bunch of svn URLs of issues:
cat list.txt
//svn.server.address/repos/project/module1/branches/issue-001-name1
...
1
vote
2answers
83 views
Parsing string by awk and get only elements without pipes or semi colons
I would like get this string line
AUGUSTYN|Stanisław|3589238
without | and :
I tried something like that:
cat baza|grep "AUGUSTYN" -n|awk -F '|' '{print $1,$2,$3,$4}'`
(baza is my file with ...
3
votes
1answer
58 views
No output from inotifywait | awk
I'm attempting to use part of a one-liner found here: Script to monitor folder for new files?
When I try the following procedure I get no output whatsoever and I cannot figure out why.
In terminal ...
1
vote
2answers
71 views
Help inserting a new line of text after matching a line of text (sed)?
I need to insert a hidden HTML input tag into any form tag within a bunch of HTML files. I assume this is possible with sed, but need help forming the command.
My idea is to search for any instance ...
2
votes
2answers
76 views
Awk: expanding first field along the column
I have a task which requires some awk scripting:
Lets say I have a file:
A,under1
,under2
,under3
,under4
,under5
B,new1
,new2
,new3
C,sm1
,sm2
How could I make it look like this using awk?
...
0
votes
2answers
49 views
Errors when searching for special characters in awk
How can I search for a pattern in awk if the text contains special characters?
billinAddress ||
when I search, it is showing an error on pipe sign ||. How can I search for the pipe sign || in awk?
...
2
votes
2answers
189 views
Data pivot with awk
I'm trying to pivot a file using awk. This is an example of my input File:
VarName;TimeString;VarValue;Validity;Time_ms
A;23.11.201215:03:53;1;1;41236627696,7593
...
1
vote
1answer
140 views
Validate Unix Timestamp using BASH
My google-fu has failed me so I'm hoping someone here can point me in the right direction. I'm in the middle of doing some data validation efforts and I suspect the system is recording some invalid ...
3
votes
4answers
246 views
removing the first and the last character of every line from command line
I am trying to remove the first and the last characters of everyline in a text file and save the resulting truncated version in a new file. Does anyone have an idea about how to do that efficiently ...
0
votes
2answers
172 views
Replace matches with multiline string using sed [duplicate]
Possible Duplicate:
Insert a multiline string into another string
I have a file called foo.txt containing:
foo
bar
baz
I want to replace every occurence of bar with a multiline string, ...
1
vote
3answers
190 views
linux/solaris + verify duplicate valid IP address from file
what the best way to find duplicate IP from file ( I have ksh script in this script I need to write function that check for duplicate IP )
for example if IP - 192.1926.23.52 exists twice in file - ...
2
votes
1answer
192 views
Bash variable in Awk script
I have this flat file database(ff_servers.db) with following contents:
192.168.154.2 Alaska hp
192.168.157.3 Colorado dell
192.168.156.3 hawaii hp
From command line, I could run:
awk ...
3
votes
1answer
57 views
Are there awk versions that provide syntax for computing aggregations?
From time to time I find myself writing awk scripts that compute some simple statistics. For example computing a histogram, the average of a value, the standard deviation or even the variance ...
...
1
vote
1answer
112 views
Sed and Awk learning material [closed]
I'm looking for some learning material for sed and awk, can anyone make any suggestions?
Please don’t refer to man and help pages.
-O-
2
votes
1answer
89 views
Using awk, how to get desired output from multiple file sources
I have two files:
file1:
A,D
B,E
C,F
file2:
G,H,I
J,K,L
Using awk, I want to print:
A,D
B,E,I
C,F,L
How can I use awk to achieve this?
1
vote
1answer
72 views
how to add a description in footer
I enjoy learning how to modify files with awk, sed, and bash. However, I know it could cause issues down the road and I would like to be prepared with a log inside the file. I was thinking about how ...
0
votes
1answer
54 views
Why is this variable not getting passed to awk? [duplicate]
Possible Duplicate:
external variable in awk
How do I pass this variable below?
This doesn't work:
fname=testfile.txt
lsof | awk '/deleted/&&/$fname/ {print $4}' *----no output*
...
1
vote
5answers
149 views
Replacing lines in files with file contents
I have several files which contain some PHP includes and I want to substitute them with the file contents. The file looks like
foo
<?php
include("file1.php");
?>
bar
baz
<?php
...
8
votes
3answers
203 views
Process last line first using awk
I have a data file that I want to normalize using awk, based on the last datapoint. Therefor, I would like to access the last data point first, to normalize the data, then process normally.
The ...
2
votes
3answers
154 views
How can I call bc from within awk in this case?
OK, so I love my awk, but yeah, it has precision issues, and unfortunately there's no easy way for me to install the multi-precision extensions known as gawkextlib.
What I am doing is, I am working ...
2
votes
3answers
190 views
Awk to output read file
I havent stepped into awk yet but it is on my plate for learning this coming year. I asked a previous question that led me to awk and it works. However, I am unsure how to input all file types and ...
3
votes
1answer
188 views
Awk - how to compare two numbers?
This example works fine:
awk -v num1=5999 -v num2=5999 'BEGIN{ print (num2==num1) ? "equal" : "not equal" }'
equal
This example does not work well:
awk -v num1=59558711052462309110012 -v ...
0
votes
1answer
146 views
If Else in Gawk
I have an example:
echo $(seq 9) | awk '{for(i=1;i<=NF;i++) if(i!=5&&i!=6) {printf "%s ",i}else{printf "%s ",""} print ""}'
1 2 3 4 7 8 9
Is it possible to use the shorthand of this ...
2
votes
2answers
181 views
print a file in hex using awk
I have a file name x
$ cat x
1A34532112345
I wants to print in hex using awk I written a script (with help from web)
fold -1 /home/cscape/Desktop/x | gawk '{ printf("%s , %X\n",$0, int($0) )}'
...
3
votes
1answer
100 views
Split file based on a pattern with leading zeros
I have a book in text format. I would like to split the book into several files where each file contains a single chapter. Therefore I'm using the following command:
awk '/Chapter/{i++}{print > ...
