current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
up vote 3 down vote favorite

I'll appreciate your help with the following issue:

I'm trying to set an array which contains a variable as part of the array name, example: Arr_$COUNTER (where $COUNTER is changed based on a loop count)

Every possible way I have tried came up with an error, such as "bad substitution" or "syntax error near unexpected token"

Here is the entire flow:

  1. There is one file which contain multiple lines. each line has 6 values separated by space

    10 20 30 40 50 60  
100 200 300 400 500 600

  2. The script, is meant to read each line from the file, and declare it as an array (with the line number which is the variable.

  3. as a test, each value should be printed and eventually another function will be executed on each value.

    #!/bin/bash
COUNTER=1
LINES=`wc -l VALUES_FILE.txt | awk '{print $1}'`
echo "Total number of lines "$LINES
echo
while [ $COUNTER -le $LINES ]
do
echo "Counter value is $COUNTER"
field=`awk "NR == $COUNTER" VALUES_FILE.txt`
echo "Field = $field"
declare -a "arr$COUNTER=($field)"
echo "arr$COUNTER[0] = ${arr$COUNTER[0]}"
echo "arr$COUNTER[1] = ${arr$COUNTER[1]}"
echo "arr$COUNTER[2] = ${arr$COUNTER[2]}"
echo "arr$COUNTER[3] = ${arr$COUNTER[3]}"
echo "arr$COUNTER[4] = ${arr$COUNTER[4]}"
echo "arr$COUNTER[5] = ${arr$COUNTER[5]}"
let COUNTER=COUNTER+1
echo
done
echo "The End"
echo

Here is the result:

Total number of lines 2

Counter value is 1
Field = 10 20 30 40 50 60
./sort.sh: line 12: arr$COUNTER[0] = ${arr$COUNTER[0]}: bad substitution
The End

What should be changed / fixed in order to have it working properly?

thank !

share|improve this question
up vote 0 down vote accepted

Some ideas:

  1. A "parameter expansion" of a variable value (the ${...} part):

    echo "arr$COUNTER[0] = ${arr$COUNTER[0]}"

    will not work. You may get around by using eval (but I do not recommend it):

    eval echo "arr$COUNTER[0] = \${arr$COUNTER[0]}"

    That line could be written as this:

    i="arr$COUNTER[0]"; echo "$i = ${!i}"

    That is called indirection (the !) in Bash.

  2. A similar issue happens with this line:

    declare -a "arr$COUNTER=($field)"

    Which should be split into two lines, and eval used:

    declare -a "arr$COUNTER"
eval arr$COUNTER\=\( \$field \)

    Again, I do not recommend using eval (in this case).

  3. As you are reading the whole file into the memory of the shell, we may as well use a simpler method to get all lines into an array:

    readarray -t lines <"VALUES_FILE.txt"

    That should be faster than calling awk for each line.

An script with all the above could be:

#!/bin/bash
valfile="VALUES_FILE.txt"

readarray -t lines <"$valfile"             ### read all lines in.

line_count="${#lines[@]}"
echo "Total number of lines $line_count"

for ((l=0;l<$line_count;l++)); do
    echo "Counter value is $l"             ### In which line are we?
    echo "Field = ${lines[l]}"             ### kepth only to help understanding.
    k="arr$l"                              ### Build the variable arr$COUNTER
    IFS=" " read -ra $k <<<"${lines[l]}"   ### Split into an array a line.
    eval max=\${#$k[@]}                    ### How many elements arr$COUNTER has?
    #echo "field $field and k=$k max=$max" ### Un-quote to "see" inside.
    for ((j=0;j<$max;j++)); do             ### for each element in the line.
        i="$k[$j]"; echo "$i = ${!i}"      ### echo it's value.
    done
done
echo "The End"
echo

However, still, AWK may be faster, if we could execute what you need in AWK.

A similar processing could be done in awk. Assuming the 6 values will be used as an IP (4 of them) and the other two are a number and an epoch time.

Just a very simple sample of an AWK script:

#!/bin/sh
valfile="VALUES_FILE.txt"
awk '
NF==6 { printf ( "IP: %s.%s.%s.%s\t",$1,$2,$3,$4)
        printf ( "number: %s\t",$5+2)
        printf ( "epoch: %s\t",$6)
        printf ( "\n" )
    }
' "$valfile"

Just make a new question with the details.

share|improve this answer
    
Thank you for the detailed information - that helped ! unfortunately the "readarray" is not working for me .. – AlonCo Feb 17 at 11:10
    
@AlonCo Assuming readarray is not available because of your bash version, then take a look to the detailed analysis in here. I am sure you will find a solution there. – user79743 Feb 18 at 4:28
    
my script is completed thanks to you and as you already guessed it's SLOW. You mentioned AWK can be faster, how? The script eventually will be used to read a large file ( 100k lines) where each line has 6 values and convert these values to IP / number/ epoch time. – AlonCo Feb 18 at 5:07
    
@AlonCo What is the intended processing needed for each file line?. If that could be done in AWK, then an awk script will run quite faster. Could you reveal the intended processing or raise a new question with the details? – user79743 Feb 18 at 6:26
    
@AlonCo I added an AWK script to only print the values to my answer. Take a look, time it. It does not read the whole file into memory (which should be an additional plus IMO). – user79743 Feb 18 at 6:58
up vote 2 down vote

You can use the variable indirection, if you assign both the name and index to a variable:

s="arr$COUNTER[0]"
echo "arr$COUNTER[0] = ${!s}"
share|improve this answer
    
I'm not sure I get it. Can you explain further? – AlonCo Feb 15 at 22:39
    
Variable indirection ${!s} works with arrays only if you use both the array name and index. See "variable indirection" under "Parameter Expansion" in man bash. – choroba Feb 15 at 23:04
    
thanks ! much appreciated. – AlonCo Feb 17 at 11:20
up vote 1 down vote

You can generate names using eval, e.g.,

eval declare -a '"arr'$COUNTER'=($field)"'

essentially quoting all of the meta-characters except the ones you want to evaluate.

So... if $COUNTER is 1, your script would do

declare -a "arr1=($field)"

Further reading:

share|improve this answer
    
This might work... I'll test it and update. Thanks?! – AlonCo Feb 15 at 22:38
    
eval is POSIX (works "everywhere"), but keep in mind that all of the generated names would get similar treatment. Readability of the script is a problem with either eval or variable indirection. – Thomas Dickey Feb 15 at 22:42
    
thanks ! much appreciated – AlonCo Feb 17 at 11:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.