Highest Voted &#39;bash awk&#39; Questions

22

votes

8answers

29k views

Is there a way to get the min, max, median, and average of a list of numbers in a single command?

I have a list of numbers in a file, one per line. How can I get the minimum, maximum, median and average values? I want to use the results in a bash script. Although my immediate situation is for ...

asked May 25 '11 at 4:42

Peter.O
9,43843275

19

votes

11answers

3k views

Remove duplicate $PATH entries with awk command

I am trying to write a bash shell function that will allow me to remove duplicate copies of directories from my path environment. I was told that it is possible to achieve this with a one line ...

bash shell awk path

asked Jun 14 '12 at 4:22

Johnny Williem
10325

16

votes

3answers

4k views

Piping from grep to awk not working

I am trying to grep the ongoing tail of file log and get the nth word from a line. Example file: $ cat > test.txt <<EOL Beam goes blah John goes hey Beam goes what? John goes forget it Beam ...

bash grep rhel awk tail

asked Aug 30 '12 at 12:04

Belmin Fernandez
5812722

11

votes

7answers

5k views

How to print the longest line in a file?

I'm looking for the simplest method to print the longest line in a file. I did some googling and surprisingly couldn't seem to find an answer. I frequently print the length of the longest line in a ...

bash awk filter

asked Nov 13 '11 at 0:58

dr.bunsen
22438

11

votes

1answer

2k views

Making a progressbar with “dialog” from rsync output

I'm looking for a way to filter/redirect rsync output in a manner where it can be fed to the "dialog --gauge" command, so I can get a nice looking progressbar during file sync. Currently I have only ...

linux bash scripting awk rsync

asked Aug 6 '12 at 11:40

chr1573r
585

10

votes

2answers

756 views

Script to quickly test all keyboard keys

I need to check some notebooks for bad keyboard keys, and so I'd like to speed that up as much as possible. I found nothing for this specific task, so my idea is a script that reads the pressed keys ...

bash grep awk keyboard stdout

asked Sep 16 '13 at 14:38

mdrg
1537

9

votes

8answers

1k views

How to remove multiple newlines at EOF?

I have files that end in one or more newlines and should end in only one newline. How can I do that with Bash/Unix/GNU tools? Example bad file: 1\n \n 2\n \n \n 3\n \n \n \n Example corrected ...

bash sed awk ed

asked Jul 4 '13 at 0:20

Bengt
314311

9

votes

3answers

3k views

How to sort the string which combined with string + numeric using bash script?

This is the data what I want to sort. But sort treats the numeric to string, the data it no sorted as I expected. /home/files/profile1 /home/files/profile10 /home/files/profile11 ...

bash shell-script sed awk sort

asked Jun 26 '12 at 10:27

user760548
4814

7

votes

5answers

324 views

SED : Case Match Pattern Replacement

I have a source code spread across several files. It has a pattern abcdef which I need to replace with pqrstuvxyz. The pattern could be Abcdef (Sentence Case) then it needs to be replaced with ...

bash text-processing sed awk patterns

asked Apr 20 at 18:17

user1263746
21337

6

votes

5answers

221 views

Strategy to extract movies's name from this ununiform dataset?

I am working on a movie database problem to improve regular expressions, this is the problem I'm running into. My dataset looks like this: Movie Name (variable space and tabs) year Movie1(can ...

bash sed awk grep regular-expression

asked Jul 4 at 18:59

Dude
1715

6

votes

2answers

494 views

How would I sort these directory names numerically?

I'm aware that I can somehow sort this output numerically (so cpu1/ follows cpu0/) ... I could probably get something to work eventually by splitting up the string various ways with awk, but is there ...

bash sed awk sort

asked Oct 27 '11 at 5:31

twblamer
319210

6

votes

4answers

7k views

Delete range of lines above pattern with sed (or awk)

I have the following code that will remove lines with the pattern banana and 2 lines after it: sed '/banana/I,+2 d' file So far, so good! But I need it to remove 2 lines before banana, but I can't ...

bash sed awk

asked Jan 24 '12 at 16:09

Teresa e Junior
1,318621

6

votes

5answers

7k views

Show sum of file sizes in directory listing

The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current ...

bash shell-script awk ls

asked Apr 16 '13 at 18:38

MattDMo
554418

5

votes

3answers

848 views

Is there a more elegant way to count words and assign that count to variables?

I have a script: #!/bin/bash /root/xiotech status > xiostatus.tmp SyncCount=$(grep -c Sync xiostatus.tmp) PauseCount=$(grep -c paused xiostatus.tmp) CopyingCount=$(grep -c Copying xiostatus.tmp) ...

bash shell-script awk

asked Feb 8 '12 at 0:48

LVLAaron
3982423

5

votes

2answers

113 views

Is there a way to make this one-liner faster?

Context I have a directory of thousands of zip files that are dated in the form YYYYMMDD_hhmmss.zip and each about 300K. Within each zip file is about 400 xml files each about 3K. The problem I ...

bash shell-script awk grep xargs

asked 2 days ago

fifosine
262

5

votes

5answers

408 views

Pattern matching across multiple lines

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output? I was using something like the following: awk ...

bash sed grep awk

asked Nov 14 '13 at 16:56

Tim Brigham
442311

5

votes

2answers

405 views

Use a script parameter in awk

Here is my script (to find the files that contain a specified pattern) : find . -type f -exec awk -v vawk="$1" '/'"$vawk"'/ {c++} c>0 {print ARGV[1] ; exit 0 } END { if (! c) {exit 1}}' \{\} \; ...

bash awk

asked Oct 5 '12 at 18:26

Nicolas
1756

5

votes

1answer

2k views

Fast elimination of duplicate lines across multiple files

I have a huge amount of data in which each (data-)line should be unique. There are a lot of files in one folder in which this is already true. It is about 15GB splitted into roughly 170 files with ...

bash files awk performance

asked Sep 11 '12 at 10:29

stefan
332314

5

votes

3answers

725 views

extracting “tag” information from git with a shell script

The vc bundle is a neat little package that extracts information about a git repo for easy insertion into a LaTeX document. It doesn't currently extract information about whether the current commit is ...

bash awk git

asked Dec 21 '10 at 13:33

Seamus
4931619

4

votes

3answers

969 views

Summing up an array inside of awk?

I have the following piece of code: sum1= sum2= declare -a a echo $temp | awk '{split($0,a,","); name=a[1] ; for(i=2;i<=4;i++) sum1+=a[i] ; for(i=5;i<=7;i++) sum2+=a[i] }' This code is ...

bash shell shell-script scripting awk

asked Sep 16 '13 at 15:10

user2179293
9629

4

votes

2answers

320 views

How can I get awk to print both matches and non-matches?

Basically, all I want to do is alter outputs if the input matches a filter, but the part being altered isn't what is being filtered (or I would use sed). The problem so far is that my awk is only ...

bash awk

asked Jun 29 '12 at 4:27

Ken Oh
525

4

votes

3answers

183 views

How to only output lines that are younger than 3 months?

cat a.txt 1387111124 ./asfjlasdf.txt 1348681215 ./akdfyxcv.txt So I need this: cat a.txt | SOMEMAGIC 1387111124 ./asfjlasdf.txt The output should only have the lines that are younger than 3 ...

bash sed awk

asked Dec 15 '13 at 13:36

gasko peter
661245

4

votes

2answers

3k views

Working with columns - awk and sed

I am trying to parse a text file that is generated by an expec script that pulls down some information from a switch. Here is a sample output: 192 0000.0000.0000 1/g23 Dynamic ...

bash sed awk

asked Jul 25 '12 at 22:34

bourne
6717

4

votes

4answers

108 views

Combine fdisk and df in the same output

sudo fdisk -l gives : 255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size ...

bash awk disk-usage fdisk

asked May 28 at 17:27

Nadina
825

4

votes

2answers

259 views

Merge fields in a file

I have a file with 7 columns, a GFF file having chromosomal regions.I want to collapse the rows where REGION ="exon" to only one row in the file.The row has to be collapsed on the basis of regions ...

bash shell sed awk bioinformatics

asked Mar 21 at 17:42

Ron
1037

4

votes

2answers

135 views

How can I delete all unordered lines from a text file?

Overview Consider an ordered list interspersed with unordered elements, e.g.: Alligator Ant Falcon <-- Baboon Badger Armadillo <-- Caiman Cat How can this list be processed so that all ...

bash text-processing sed awk

asked Feb 27 at 17:42

Glutanimate
371222

4

votes

1answer

122 views

Looking for lines which is in one file but not in other using Unix and Awk

I have 2 files with 7 fields and I want to print the lines which are present in file1 but not in file2 based on field1 and field2. Logic: I want to print all the lines, where there is a particular ...

bash text-processing awk

asked Aug 21 '13 at 7:01

Namrata
1699

4

votes

2answers

1k views

recursive search for a pattern, then for each match print out the specific SEQUENCE: line number, file name, and no file contents

What I am after is almost exactly the same as can be found here, but I want the format "line number, separator, filename, newline" in the results, thus displaying the line number at the beginning of ...

bash find grep awk

asked Oct 16 '13 at 23:29

John Sonderson
1364

4

votes

2answers

508 views

Replace one line with STDIN in a Makefile

One Makefile target is supposed to be in charge of: grabbing the output of $ perl Markdown.pl src/index.md (markdown to html) using that string to replace one line, something like CONTENT ...

bash sed awk make

asked Jan 29 '12 at 19:24

chelmertz
1235

4

votes

1answer

677 views

Pass colors from ls through pipe to awk 'print' statement

This is a follow-up to my question from yesterday, Show sum of file sizes in directory listing. Thanks to Zero Piraeus and a point in the right direction by Mauritz Hansen, I now have function ...

bash awk ls pipe colors

asked Apr 17 '13 at 17:47

MattDMo
554418

4

votes

6answers

340 views

What is the easiest way to check if column A and B values goes both ways with AWK? [closed]

What is the easiest way to check if column A and B values goes both ways? Output to check: Mike John John Mike Pamela Barbara Barbara Pamela Mike Paul Roger Paul Desired output Mike <-> ...

bash text-processing awk python perl

asked Sep 12 '12 at 6:39

tiny
10817

3

votes

2answers

197 views

Bash Script To Find Debian Release Number from /etc/debian_version

The following is what I'm currently using for testing the version of Debian. Instead of doing it this way, can I make it more succinct? #!/bin/bash VERSION=NONE if [ $(cat /etc/debian_version) == ...

bash debian awk bash-script

asked Jan 19 at 2:19

Danijel J
1939

3

votes

2answers

85 views

Command in variable one are affecting output of other variable when displayed together in UNIX

I am having two file naming Extra_info and Class_info Class_info contains data : stuId stuName stuClass S01 jagdeep First S02 harsh First S03 Roop Second S04 Kali Third ...

bash awk echo cut

asked Feb 19 at 16:18

Jagdeep
1263

3

votes

4answers

393 views

Align a hex text file at 9 bytes each line

I want to align the selected portion of a text file at 9 bytes each column. For example suppose my text file looks like below. 00 2f c6 b8 29 fd 02 37 11 00 9f 74 34 0b 60 72 38 20 00 9e 61 33 8e ...

bash sed awk python

asked Jan 25 '12 at 21:25

liv2hak
1185

3

votes

3answers

144 views

Replace multiple strings in a single pass

I'm looking for a way to replace placeholder strings in a template file with concrete values, with common Unix tools (bash, sed, awk, maybe perl). It is important that the replacement is done in a ...

bash sed awk string replace

asked Jun 18 at 22:18

Ambroz Bizjak
20126

3

votes

2answers

285 views

How can I print out all but the last n characters in bash?

I'm working with standard input (cating two files together) and want all but the last 10 characters of the result to be passed through the next command. All I can find through searching is using sed, ...

bash shell sed awk tr

asked Apr 20 at 9:27

Ehryk
296111

3

votes

6answers

2k views

Text file look-up by column

I have a file in this format: [#] OWNER_NAME NAME SIZE [6] Robottinosino Software 200 [42] Robottinosino Ideas worth zero 188 [12] ...

bash shell-script sed awk filter

asked Jun 25 '12 at 7:22

Robottinosino
775625

3

votes

3answers

342 views

swapping fields in columns by fixed pattern (yes, no, no, yes)

I want to produce a following list of pages (for imposition work); (number of pages is variable, but I make a sample of my needs using a list of 16 pages, logic is the same for longer lists) 1,16 8,9 ...

bash awk numbering

asked Apr 15 '12 at 22:07

Dingo
1304

3

votes

2answers

303 views

Add thousands separator in a number

In python re.sub(r"(?<=.)(?=(?:...)+$)", ",", stroke ) To split a number by triplets, e.g.: echo 123456789 | python -c 'import sys;import re; print re.sub(r"(?<=.)(?=(?:...)+$)", ",", ...

bash shell-script awk string

asked Feb 6 at 6:48

user2496
385

3

votes

1answer

1k views

Make awk use bash with the system() command

Is there a way to make awk use bash instead of sh when running system commands using the system() call? I want to use some bash-specific features such as [[ ]] and < > string comparison operators: ...

bash shell awk command

asked Jun 16 '12 at 18:02

haelix
1736

3

votes

2answers

44 views

awk: Blank line with date

I use the following code to write the online users' process numbers and the date they were checked: w | tail -n +3 | awk ' { "date +%H:%M:%S" | getline tim} {if ($1 != "") ...

bash awk

asked Jun 26 at 22:50

user71884
494

3

votes

3answers

3k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...

bash sed awk perl

asked Aug 25 '13 at 3:37

user177073
3614

3

votes

2answers

212 views

Automated partition selection in bash script

I'm trying to write a bash script to run from Linux on a CD or flashdrive that can mount Windows and change stuff, but I don't know how to go about detecting which partition/drive is Windows. My best ...

bash awk windows devices

asked Apr 22 '13 at 14:39

Peace Blaster
14318

3

votes

2answers

822 views

simple script rotating table

I'm trying to write a simple script, that just rotates csv table. I mean I have some file: head1;head2;head3 field11;field12;field13 and so on. All I want, just to make my file ...

bash sed awk

asked Mar 26 '12 at 15:08

rush
11.4k2654

2

votes

3answers

805 views

How to truncate the second column from a dot (.)?

INPUT: 10.175.192.16 vgxyxgf.de.foo.net asdf 34efg 10.175.192.17 sdtds.de.foo.net xyyxv e5432 10.175.243.20 asdfycfb.us.foo.net dfvasf 34525 OUTPUT: 10.175.192.16 vgxyxgf ...

bash awk

asked Jan 26 '12 at 4:13

LanceBaynes
2,1951789188

2

votes

4answers

2k views

Increment number in bash variable string

VERSION="4.0beta12" NEXT_VERSION=????? Writing an automatic release script, I have the current version in a variable $VERSION, and I would like to calculate the next version as $NEXT_VERSION. Rule: ...

bash awk arithmetic

asked Oct 24 '11 at 4:43

Nicolas Raoul
8661928

2

votes

3answers

3k views

Extracting tokens from a line of text

Using bash scripting and grep/awk/sed, how can I split a line matching a known pattern with a single character delimiter into an array, e.g. convert token1;token2;token3;token4 into a[0] = token1 … ...

bash grep text-processing sed awk

asked Feb 22 '11 at 11:17

Jas
17528

2

votes

3answers

1k views

Easy way to parse syslog date format

I'd like to write a script that analyses the last week's syslog. But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial. Is there a way ...

bash sed awk perl syslog

asked May 22 '13 at 11:00

artfulrobot
416313

2

votes

3answers

888 views

Bash commands/script to remove a line from CSV with duplicate in column

I have a lot CSV files that I have combined. However, there are duplicates, but the entire line is not duplicated. I do have a column that I want to use as the criteria to search for a duplicate. ...

bash sed awk

asked Mar 22 '13 at 16:02

Peaceful_Warrior
111

2

votes

3answers

2k views

Trouble getting awk output in for loop

I'm trying to create a script that will check a website for a word. I have a few to check so I'm trying to input them via another file. The file is called "testurls". In the file I list the keyword ...

bash shell text-processing awk

asked Nov 24 '11 at 21:52

jetgerbil
134

your communities

Tagged Questions

Related Tags