current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.
up vote 0 down vote favorite

Bash newbie here!

I have a bash script that for each file in a set, greps each line in the file for a string. Then it splits the line on commas, converts the 7th element to a float, and increments a running total by that value.

It looks like this:

for filename in data*.CSV; do
   echo $filename
   ACTUAL_COST=0
   grep '040302010' $filename | while read -r line ; do
       IFS=',' read -a array <<< "$line"
       ACTUAL_COST=$(echo "$ACTUAL_COST + ${array[7]}" | bc)
       echo $ACTUAL_COST
   done
   echo $ACTUAL_COST
done

But the problem I'm having is that this produces output like this:

53.4
72.2
109.1
0

The last value is always 0. After Googling a bit, I think this is because the while loop is executing in a subshell, and so the outer variable isn't changed.

I understand that I probably need to execute the inner loop in a function, but I'm struggling to work out how to do this. Can anyone help?

share|improve this question
    
Related: Why is using a shell loop to process text considered bad practice? –  Stéphane Chazelas Jun 29 at 15:11

3 Answers 3

up vote 0 down vote accepted

It might be handy to introduce another command substitution, with the core loop logic defined in a function:

sum_cost() {
   sum=0
   while read -r line ; do
       IFS=',' read -a array <<< "$line"
       sum=$(echo "$sum + ${array[7]}" | bc)
   done
   echo $sum
}

for filename in data*.CSV; do
   echo $filename
   ACTUAL_COST=$(grep '040302010' $filename | sum_cost)
   echo $ACTUAL_COST
done
share|improve this answer
up vote 2 down vote

That's not how you do shell scripting. You're running several commands in sequence for each line of the files!

Here you want something like:

awk -F, '/040302010/ {actual_cost += $7}
         ENDFILE {print FILENAME ":", +actual_cost; actual_cost=0}
        ' data*.CSV

(assuming GNU awk).

That's one command in total for all your files.

share|improve this answer
up vote 0 down vote

To avoid the subshell you can use the following:

while read -r line
do
    your_stuff
done < <(grep '040302010' $filename')

That way you are able to fill in the result(s) into the variable.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.