When writing a
bash shell script it is important to understand how
IFS (Internal Field Separtor) works and how it can be modified/changed.
What is IFS
As mentioned in the
IFS is defined as follows:
IFS The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``<space><tab><newline>''.
IFS defines which characters
bash will use to delimit words. Naturally whitespace is the default,
<space><tab><newline>. Of course, as is usual in the English language, the whitespace is not considered part of the word and is ignored.
To access the
IFS characters we just need to print out the characters in the variable
$IFS. As mentioned in Bash Backslash Escaped Characters and Whitespace Explained with Examples the backslash escaped characters are different from what they represent i.e.
<newline> is not the same as
\n. The default
IFS characters are
<space>tn. This will be proven in the following examples.
IFS, by default, is just whitespace we won’t really see much when we try to print it out:
[ahmed@amayem ~]$ echo hello $IFS world hello world
As we can see,
hello world is printed without a
<newspace> in between. The reason for this is simple. After
$IFS was expanded to
bash used word splitting on the whole line. So
bash had to split the following line:
So after word splitting,
echo had only two arguments:
bash pages have more to add:
echo [-neE] [arg ...] Output the args, separated by spaces, followed by a newline.
Notice how the characters in
IFS are not considered part of the word.
[ahmed@amayem ~]$ echo 'hello $IFS world' hello $IFS world
This happened because single quotes take away the special meaning of the
$, as explained here.
[ahmed@amayem ~]$ echo "hello $IFS world" hello world
This time it worked because the double quotes don’t take away the meaning of the
$IFS was expanded. The double quotes preserve the whitespace and prevents word splitting here as mentioned here.
Viewing IFS as Backslash Escaped Characters
Now that we have established that
IFS is not the backslash escaped characters, we face the problem that we can’t see the whitespace. It would be nice if we could convert the whitespace to backslash escaped characters so we know what they are. The following methods work:
printf %q “$IFS”
[ahmed@amayem ~]$ printf %q "$IFS" $' tn'[ahmed@amayem ~]$
%q option is a cool feature of
printf as mentioned in the
%q causes printf to output the corresponding argument in a format that can be reused as shell input
Notice that I had to use the double quotes so that the whitespace appears. Using single quotes or no quotes would not work as mentioned earlier.
[ahmed@amayem ~]$ echo -n "$IFS" | cat -te ^I$
cat has options that allow for whitespace characters to be represented with something else as mentioned in the
-t Display non-printing characters (see the -v option), and display tab characters as `^I'. -v Display non-printing characters so they are visible. Control characters print as `^X' for control-X; the delete character (octal 0177) prints as `^?'. Non-ASCII characters (with the high bit set) are printed as `M-' (for meta) followed by the character for the low 7 bits.
So we will have to translate
-n flag of echo removes the final newline that is usually added by
[ahmed@amayem ~]$ echo -n "$IFS" | od -abc 0000000 sp ht nl 040 011 012 t n 0000003
First we piped the
IFS characters to
od, but without the last newline that
echo adds, by adding the
-n flag. Then
od has the following options as explained in its
-a Output named characters. Equivalent to -t a. -b Output octal bytes. Equivalent to -t o1. -c Output C-style escaped characters. Equivalent to -t c.
So really the one we want is the
c, but we won’t see the space very clearly so it is good to have the other two as well.
set | grep IFS
Let’s not forget that
IFS is a shell variable after all, hence we should be able to see the characters of
IFS with all the other shell variables. Shell variables can be seen using
Without options, the name and value of each shell variable are displayed in a format that can be reused as input for setting or resetting the currently-set variables.
Of course since there are so many shell variables we will pipe the output into
grep and filter for
[ahmed@amayem ~]$ set | grep IFS IFS=$' tn'
set is conveniently printing out the value in a way that can be used to set the variables, it does not mean that the characters of
tn. They are
<space><tab><newline> as explained earlier
IFS and using whitespace characters we have to be careful how we quote the new string. If we want to use backslash escaped characters then we have to use the form
$'string' as mentioned here. If we want to use the actual characters then we have to use single or double quotes.
Modifying Using the Form $’string’
[ahmed@amayem ~]$ IFS=$'n'; printf %q "$IFS" $'n'
Modifying Using Single or Double Quotes
[ahmed@amayem ~]$ IFS=" > "; printf %q "$IFS" $'n' [ahmed@amayem ~]$ IFS=' > '; printf %q "$IFS" $'n'
Backing up the Default
Whenever we change the IFS it is usually a good idea to save the default first and then restore the
IFS to its default after you are done:
[ahmed@amayem ~]$ defaultIFS=$IFS; IFS=$'n'; printf "%qn" "$IFS"; IFS=$defaultIFS; printf "%qn" "$IFS" $'n' $' tn'
Restoring the Default
[ahmed@amayem ~]$ IFS=$' tn'