A common case that comes up when programming is accessing a variable that is received as a parameter. Bash scripting accomplishes this using something called indirect expansion, which will be our topic in this post.
Setup
Let’s make a shell script. In your favourite editor type
#!/bin/bash
x=2
letters=(a b c d)
And save it somewhere as indirect.sh. Now we need to make it executable as follows:
[ahmed@amayem ~]$ chmod +x ./indirect.sh
[ahmed@amayem ~]$ ./indirect.sh
Looks good so far.
Indirect Expansion
Let’s take a look at the man pages:
${parameter}
The value of parameter is substituted. The braces are required when parameter is a positional parameter with more than one digit, or when parameter is followed by a character which is not to be interpreted as part of its name.
If the first character of parameter is an exclamation point, a level of variable indirection is introduced. Bash uses the value of the variable formed from the rest of parameter as the name of the variable; this variable is then expanded and that value is used in the rest of the substitution, rather than the value of parameter itself. This is known as indirect expansion.
Now to apply it.
Simple Variable
Accessing the Variable
First let’s try to specify a simple variable in this case x:
fn()
{
echo ${!1}
}
fn x
Running it gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
2
Excellent. What does, ${1} mean? It indicates the first parameter passed to the function. It is another way of writing $1. When we put the exclamation mark in the beginning, the !1 is replaced with the value of $1, so it becomes ${x}. So in effect ${!1} is ${x}.
Changing the Variable
I would like to get the name of the variable as a parameter and then change the value of that variable in the function. Let’s try the following:
fn()
{
${!1}=5
echo $x
}
fn x
It gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
./../arrays.sh: line 12: 2=5: command not found
2
So it seems that indirect expansion can’t be used as a pointer. When ${!1} is reached, the value of the variable is replaced, so in our case it was like saying 2=5, which doesn’t make sense. Instead let’s try using ${1}, which would translate to x=5.
fn()
{
${1}=5
echo $x
}
fn x
However, we still get this:
[ahmed@amayem ~]$ ./indirect.sh
./../arrays.sh: line 12: x=5: command not found
2
So it seems that ${1} is substituted with x but the resulting string x=5 is treated as a command name. We need the string x=5 to be evaluated. This needs the eval command:
Eval
Let’s see what the bash man pages have to say about eval:
eval [arg ...]
The args are read and concatenated together into a single command. This command is then read and executed by the shell, and its exit status is returned as the value of eval. If there are no args, or only null arguments, eval returns 0.
Let’s try to apply it:
fn()
{
eval ${1}=5
echo $x
}
fn x
This give us the following:
[ahmed@amayem ~]$ ./indirect.sh
5
Success. However, care should be taken when using eval that the final string should not be something malicious.
Other Methods for assignment
There are other commands that deal specifically with assigning values to variables, such as the read and printf commands.
read
Let’s take a look at the read command entry in the bash man pages:
read [-ers] [-u fd] [-t timeout] [-a aname] [-p prompt] [-n nchars] [-d delim] [name ...]
One line is read from the standard input, or from the file descriptor fd supplied as an argument to the -u option, and the first word is assigned to the first name, the second word to the second name, and so on, with leftover words and their intervening separators assigned to the last name. If there are fewer words read from the input stream than names, the remaining names are assigned empty values. The characters in IFS are used to split the line into words. The backslash character () may be used to remove any special meaning for the next character read and for line continuation.
Let’s apply it:
fn()
{
read ${1} <<< 5
echo $x
}
fn x
The <<< 5 part is basically giving 5 as the input from standard input. When we run it we get:
[ahmed@amayem ~]$ ./indirect.sh
5
Success.
printf
As usual let’s see what the bash man pages have to say:
printf [-v var] format [arguments]
Write the formatted arguments to the standard output under the control of the format. The format is a character string which contains three types of objects: plain characters, which are simply copied to standard output, character escape sequences, which are converted and copied to the standard output, and format specifications, each of which causes printing of the next successive argument. In addition to the standard printf(1) formats, %b causes printf to expand backslash escape sequences in the corresponding argument (except that c terminates output, backslashes in ', ", and ? are not removed, and octal escapes beginning with may contain up to four digits), and %q causes printf to output the corresponding argument in a format that can be reused as shell input.
The most important part is the following:
The -v option causes the output to be assigned to the variable var rather than being printed to the standard output.
Let’s apply it and use printf‘s formatting potential:
fn()
{
printf -v ${1} "%.10i" 5
echo $x
}
fn x
The format string "%.10i" is saying print the argument as an integer which is padded with zeroes and is 10 characters long. These are the results:
[ahmed@amayem ~]$ ./indirect.sh
0000000005
Success.
Specifying an array
Accessing one element
Let’s use letters instead of x, and try accessing as we did with the simple elements before.
fn()
{
echo ${!1}
}
fn letters
We get the following:
[ahmed@amayem ~]$ ./indirect.sh
a
This is because when the name of an array is called it gives the first element as mentioned here. How can we access other elements that we may want?
fn()
{
elem2=$1"[2]"
echo ${!elem2}
}
fn letters
Give us:
[ahmed@amayem ~]$ ./indirect.sh
c
Basically ${!elem2} becomes ${letters[2]}.
Accessing all elements
fn()
{
all=$1"[*]"
echo ${!all}
}
fn letters
Gives us:
[ahmed@amayem ~]$ ./indirect.sh
a b c d
Accessing array length and indices
As mentioned here we find the length of an array with the ${#arrayname[*]} syntax and the array indices with the ${!arrayname[*]} syntax. Let’s try to get the length using indirections:
fn()
{
all="#"$1"[*]"
echo ${!all}
}
fn letters
When we try it we get an empty line. It seems that bash looked for a variable called #letters[*] and couldn’t find one so nothing was printed except the new line added by echo. The same thing happens with !letters[*].
Bash-hackers had this to say:
It was an unfortunate design decision to use the ! prefix for indirection, as it introduces parsing ambiguity with other parameter expansions that begin with !. Indirection is not possible in combination with any parameter expansion whose modifier requires a prefix to the parameter name. Specifically, indirection isn’t possible on the ${!var@}, ${!var}, ${!var[@]}, ${!var[]}, and ${#var} forms. This means the ! prefix can’t be used to retrieve the indices of an array, the length of a string, or number of elements in an array indirectly (see indirection for workarounds). Additionally, the !-prefixed parameter expansion conflicts with ksh-like shells which have the more powerful “name-reference” form of indirection, where the exact same syntax is used to expand to the name of the variable being referenced.
So there we have it. Luckily they also gave us a workaround:
Array length and indices workaround
Luckily another Bash-hackers wiki gives us a workaround. I have used the relative part here.
fn()
{
local -a 'arraykeys=("${!'"$1"'[@]}")'
echo ${arraykeys[*]}
echo ${#arraykeys[*]}
}
fn letters
Gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
0 1 2 3
4
There it is. Success. Why does local -a 'arraykeys=("${!'"$1"'[@]}")' work? I am still looking into it. If I find out why I plan to make a post about it.
UPDATE: 8th December 2017 Thanks to Rodrigo Ribeiro Gomes for supplying the following explanation in his comment:
The solution in local -a ‘arraykeys=(“${!'”$1″‘[@]}”)’ works due to a simple string concatenation:
1) The main string is delimited by single quotes. So we have this string: arraykeys=(“${!
2) Next, we concatenate the value of $1. This comes from “$1” expression. At this point we have arrayKeys=(“${!letters
3) Finally, we append the rest of the string, that start next single quote: [@]}”).
4) This result in arrayKeys=(“${!letters[@]}”)
It is the same as use declare ‘a=someValue’; echo $a;
Modifying array elements
First attempts:
At first we may be tempted to try the following:
fn()
{
${1}=5
echo ${letters[*]}
}
fn letters
But that would give us the following:
[ahmed@amayem ~]$ ./indirect.sh
./../arrays.sh: line 27: letters=5: command not found
a b c d
It seems we have to resort to eval again.
eval
fn()
{
eval ${1}=5
echo ${letters[*]}
}
fn letters
Gives us:
[ahmed@amayem ~]$ ./indirect.sh
5 b c d
Specifying an index to modify
fn()
{
eval ${1}"[2]"=5
echo ${letters[*]}
}
fn letters
Gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
a b 5 d
read
fn()
{
read ${1}"[2]" <<< 5
echo ${letters[*]}
}
fn letters
Gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
a b 5 d
printf
First element
fn()
{
printf -v ${1} "%.10i" 5
echo ${letters[*]}
}
fn letters
Gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
0000000005 b c d
Specifying an index
Let’s try the same thing as before:
fn()
{
printf -v ${1}"[2]" "%.10i" 5
echo ${letters[*]}
}
fn letters
But we get the following:
[ahmed@amayem ~]$ ./../arrays.sh
./../arrays.sh: line 15: printf: `letters[2]': not a valid identifier
a b c d
Looks like it doesn’t like it.
Workaround
We will have to use a workaround of saving the output into a temp variable and then putting the value into the array at the desired index using one of the previous methods.
fn()
{
printf -v temp "%.10i" 5
read ${1}"[2]" <<< $temp
echo ${letters[*]}
}
fn letters
Gives us the following:
[ahmed@amayem ~]$ ./indirect.sh
a b 0000000005 d
Success.
