Bash Arrays 2: Different Methods for Looping Through an Array

In the previous shell array post we discussed the declaration and dereferencing of arrays in shell scripts. This time we will take a look at the different ways of looping through an array.

Setup

This is the same setup as the previous post
Let’s make a shell script. In your favourite editor type

#!/bin/bash

And save it somewhere as arrays.sh. Now we need to make it executable as follows:

[ahmed@amayem ~]$ chmod +x ./arrays.sh 
[ahmed@amayem ~]$ ./arrays.sh 
[ahmed@amayem ~]$

Looks good so far.
Let’s declare some arrays:

#Declarations
five=5
array1[5]=$five

declare -a array2

array3=("zero" "1" "two" "3" "four")

Philosophy

In order to loop through any array, whether in shell scripting or any other language, I need to answer the following questions:

How do I know where I am in the array?
How do I know where the next element is?
How do I know when I have gone through the whole array?

We will see how each method answers these questions:

Classic for loop

Let’s see how shell script does for loops from the bash man pages.

for (( expr1 ; expr2 ; expr3 )) ; do list ; done
    First, the arithmetic expression expr1 is evaluated according to the rules described below under ARITHMETIC EVALUATION. The arithmetic expression expr2 is then evaluated repeatedly until it evaluates to zero. Each time expr2 evaluates to a non-zero value, list is executed and the arithmetic expression expr3 is evaluated. If any expression is omitted, it behaves as if it evaluates to 1. The return value is the exit status of the last command in list that is executed, or false if any of the expressions is invalid.

Looping through the array based on its length

The idea here is we find the length of the array and loop that many times through it starting from zero and adding one to the index each time. So it will look something like this:

#looping
for ((i=0; i<${#array3[*]}; i++));
do
    echo ${array3[i]}
done

It gives us:

[ahmed@amayem ~]$ ./arrays.sh 
zero
1
two
3
four

Great. But what assumptions are we making.

Assumptions and limitations

These are our assumptions:

The indices start from zero
The indices are serial, i.e. the next index is always one higher than the one before

This limits us to arrays that fulfill these assumptions. For example if we try the above loop with array1 instead we would get an empty line. The loop would execute once only because the array has one element at index 5, but the loop is looking for that element at index 0.

We need to find a better way.

Special Array for loop

bash gives us a special for loop for arrays:

for name [ in word ] ; do list ; done
    The list of words following in is expanded, generating a list of items. The variable name is set to each element of this list in turn, and list is executed each time. If the in word is omitted, the for command executes list once for each positional parameter that is set (see PARAMETERS below). The return status is the exit status of the last command that executes. If the expansion of the items following in results in an empty list, no commands are executed, and the return status is 0.

Looping through the array using built in loop

Let’s give it a try:

#looping
for element in ${array3[*]};
do
    echo $element
done

It outputs the elements as last time. Notice that I can also add more than one array to be looped over after one another, for example:

for element in ${array3[*]} ${array1[*]};

would give me:

[ahmed@amayem ~]$ ./arrays.sh 
zero
1
two
3
four
5

limitations

This way is very easy, but what if I want to know the index of the element. In that case I have a problem here because the index here is hidden. I have to use a slightly different method.

Looping through the array based on indices

This is the way with the least assumptions and limitations. We first get an array or list of indices of an array (we saw how to do that here), then loop over the indices and use the indices to access the array elements:

array3indices=${!array3[*]}
for index in $array3indices;
do
    echo ${array3[$index]}
done

array3indices=(${!array3[*]})
for ((i=0; i<${#array3indices[*]}; i++));
do
    echo ${array3[${array3indices[i]}]}
done

This last way is a lot messier than than the one before it, because the builtin array for loop hides all that messiness. Something to note here is that we had to turn the indices in the last method into an array: array3indices=(${!array3[*]}) so that we can dereference the elements, however the builtin array for loop takes a string.

array3indices=${!array3[*]}         #Gives a string of all the indices separated with IFS
array3indices=(${!array3[*]})       #Gives an array of all the indices

The builtin array for loop loops through the string automatically based on the IFS:

IFS The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``<space><tab><newline>''.

References

The bash man pages.