current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.
up vote 0 down vote favorite

Good day.

I want to parse data of particular columns from several files(aprilPlate.txt, mayPlate.txt, junePlate.txt, julyPlate.txt, augustPlate.txt) using For loop.

The data of input files (aprilPlate.txt, mayPlate.txt, junePlate.txt, julyPlate.txt, augustPlate.txt) look like:

Incl    Cal Ps  Name    Q   Con Std Status
True    255 A1  Sample 1    35.86       0   
True    255 A2  Sample 2    36.06       0   
True    255 A3  Sample 3    17.45       0   
True    255 A4  Sample 4    17.56       0   
True    255 A5  Sample 5    17.55       0   
True    255 A6  Sample 6    40.00       0   
True    255 A7  Sample 7    36.38       0   
True    255 A8  Sample 8    27.98       0   
True    255 A9  Sample 9    27.95       0   
True    255 A10 Sample 10   28.19       0   
True    255 A11 Sample 11   36.93       0   
True    255 A12 Sample 12   37.74       0   
True    255 A13 Sample 13   17.88       0   
True    255 A14 Sample 14   17.82       0   
True    255 A15 Sample 15   17.90       0   
.
.
.

By using FOR loop, I able to get the task done. However, in order to get desire data from all files, I have to modify the file name in the script (below) manually. 

#!/bin/bash

# parse the data of desire columns from target file
# rename the column name 
# redirect the stdoutput to a text file

for z in A B; 
  do for i in 3 4 5 13 14 15; 
    do grep $z$i aprilPlate.txt | 
       awk -F "\t" '{print $3 "\t" $5}' | 
       sed -e 's/A[3-5]/st_SWC/g;s/A[1][0-9]/st_SWD/g;s/B[3-5]/st_TZC/g;s/B[1][0-9]/st_TZD/g;' >> stone.txt; 
  done;
done

for z in E F; 
  do for i in 8 9 10 18 19 20;
    do grep $z$i aprilPlate.txt | 
       awk -F "\t" '{print $3 "\t" $5}' | 
       sed -e 's/E[8-9]\|E[1][0]/su_SWC/g;s/E[1][0-9]\|E[2][0]/su_SWD/g;s/F[8-9]\|F[1][0]/su_TZC/g;s/F[1][0-9]\|F[2][0]/su_TZD/g;' >> suy.txt;
  done;
done

paste -d'\t' stone.txt suy.txt >> aprilPlate.data.txt
  • My apologies, I made a mistake in my previous coding. The correction was done.

The output of parsed data file should looks like:

st_SWC  17.45   su_SWC  28.85
st_SWC  17.56   su_SWC  28.79
st_SWC  17.55   su_SWC  28.82
st_SWD  17.88   su_SWD  29.24
st_SWD  17.82   su_SWD  29.18
st_SWD  17.90   su_SWD  29.23
st_TZC  18.06   su_TZC  25.99
st_TZC  18.09   su_TZC  25.98
st_TZC  18.13   su_TZC  26.02
st_TZD  17.75   su_TZD  25.00
st_TZD  17.70   su_TZD  25.01
st_TZD  17.69   su_TZD  24.98

I would like to ask, is there anyway I can have the files as 3rd variable in the script? Other solutions are welcome.

I am not a Unix & Linux guru, thus, could someone please help or share your experience with me in this case.

Thanks for your time.

share|improve this question

2 Answers 2

up vote 0 down vote accepted

Something like will help you

for file in aprilPlate.txt  mayPlate.txt  junePlate.txt  julyPlate.txt  augustPlate.txt;
do
for z in A B; 
  do for i in 3 4 5 13 14 15; 
    do grep $z$i $file | 
       awk -F "\t" '{print $3 "\t" $5}' | 
       sed -e 's/A[3-5]/SWC/g;s/A[1][0-9]/SWD/g;s/B[3-5]/TZC/g;s/B[1][0-9]/TZD/g;' >> stone.txt; 
  done;
done
<snip>
done
share|improve this answer
    
Thanks for your suggestion. –  KJ Lim Apr 30 at 9:52
    
Of course w/o commas between filenames (my mistake, edited) –  Romeo Ninov Apr 30 at 10:10
up vote 0 down vote

Not sure exactly what you are trying to do since you have no input or output examples, but the following should work.

#!/bin/bash
awksrc='BEGIN{FS=OFS="\t"}
{ gsub(/A[345]|E[89]|E10/, "SWC");
  gsub(/A1[0-9]|E1[1-9]|E20/, "SWD");
  gsub(/B[345]|F[89]|F10/, "TZC");
  gsub(/B1[0-9]|F1[1-9]|F20/, "TZD");
}
/SW[CD]{print $3, $5 >"stone.txt"}
/TZ[CD]{print $3, $5 >"suy.txt"}'
for file in aprilPlate.txt mayPlate.txt junePlate.txt julyPlate.txt ...
do
    awk "$awkscr" $file >/dev/null
    paste -d'\t' stone.txt suy.txt >> ${file%.txt}.data.txt
done

The ${file%.txt} gets the base name, without the extension to be able to add the .data.txt extension.

This reduces the number of shell loops to just one, and processes each file once. The output of the awk script is unneeded since the files are getting written to inside it.

Again, without example input, it's hard to say if this is correct for your purposes.

share|improve this answer
    
I have the input and output examples included in my question now. Thanks for your suggestion. –  KJ Lim Apr 30 at 7:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.