Bash IFS: its Definition, Viewing it and Modifying it

When writing a bash shell script it is important to understand how IFS (Internal Field Separtor) works and how it can be modified/changed.

What is IFS

As mentioned in the man bash pages IFS is defined as follows:

IFS    The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``<space><tab><newline>''.

So basically IFS defines which characters bash will use to delimit words. Naturally whitespace is the default, <space><tab><newline>. Of course, as is usual in the English language, the whitespace is not considered part of the word and is ignored.

Viewing IFS

To access the IFS characters we just need to print out the characters in the variable $IFS. As mentioned in Bash Backslash Escaped Characters and Whitespace Explained with Examples the backslash escaped characters are different from what they represent i.e. <newline> is not the same as \n. The default IFS characters are <space><tab><newline> not <space>tn. This will be proven in the following examples.

No Quotes

Since IFS, by default, is just whitespace we won’t really see much when we try to print it out:

[ahmed@amayem ~]$ echo hello $IFS world
hello world

As we can see, hello world is printed without a <newspace> in between. The reason for this is simple. After $IFS was expanded to <space><tab><newline>, bash used word splitting on the whole line. So bash had to split the following line:

`hello<space><space><tab><newline><space>world`

So after word splitting, echo had only two arguments: hello and world. The man bash pages have more to add:

echo [-neE] [arg ...]
    Output the args, separated by spaces, followed by a newline.

Notice how the characters in IFS are not considered part of the word.

Single Quotes

[ahmed@amayem ~]$ echo 'hello $IFS world'
hello $IFS world

This happened because single quotes take away the special meaning of the $, as explained here.

Double Quotes

[ahmed@amayem ~]$ echo "hello $IFS world"
hello   
 world

This time it worked because the double quotes don’t take away the meaning of the $ so $IFS was expanded. The double quotes preserve the whitespace and prevents word splitting here as mentioned here.

Viewing IFS as Backslash Escaped Characters

Now that we have established that IFS is not the backslash escaped characters, we face the problem that we can’t see the whitespace. It would be nice if we could convert the whitespace to backslash escaped characters so we know what they are. The following methods work:

printf %q “$IFS”

[ahmed@amayem ~]$ printf %q "$IFS"
$' tn'[ahmed@amayem ~]$

The %q option is a cool feature of printf as mentioned in the man bash pages:

%q causes printf to output the corresponding argument in a format that can be reused as shell input

Notice that I had to use the double quotes so that the whitespace appears. Using single quotes or no quotes would not work as mentioned earlier.

cat -te

[ahmed@amayem ~]$ echo -n "$IFS" | cat -te
 ^I$

Like printf, cat has options that allow for whitespace characters to be represented with something else as mentioned in the man cat pages:

-t      Display non-printing characters (see the -v option), and display tab characters as `^I'.
-v      Display non-printing characters so they are visible. Control characters print as `^X' for control-X; the delete character (octal 0177) prints as `^?'. Non-ASCII characters (with the high bit set) are printed as `M-' (for meta) followed by the character for the low 7 bits.

So we will have to translate ^I to t and $ to n. The -n flag of echo removes the final newline that is usually added by echo.

od -[ab]c

[ahmed@amayem ~]$ echo -n "$IFS" | od -abc
0000000   sp  ht  nl                                                    
          040 011 012                                                    
              t  n                                                    
0000003

First we piped the IFS characters to od, but without the last newline that echo adds, by adding the -n flag. Then od has the following options as explained in its man pages:

-a          Output named characters.  Equivalent to -t a.
-b          Output octal bytes.  Equivalent to -t o1.
-c          Output C-style escaped characters.  Equivalent to -t c.

So really the one we want is the c, but we won’t see the space very clearly so it is good to have the other two as well.

set | grep IFS

Let’s not forget that IFS is a shell variable after all, hence we should be able to see the characters of IFS with all the other shell variables. Shell variables can be seen using set:

Without options, the name and value of each shell variable are displayed in a format that can be reused as input for setting or resetting the currently-set variables.

Of course since there are so many shell variables we will pipe the output into grep and filter for IFS:

[ahmed@amayem ~]$ set | grep IFS
IFS=$' tn'

Remember that set is conveniently printing out the value in a way that can be used to set the variables, it does not mean that the characters of IFS are tn. They are <space><tab><newline> as explained earlier

Modifying IFS

When modifying IFS and using whitespace characters we have to be careful how we quote the new string. If we want to use backslash escaped characters then we have to use the form $'string' as mentioned here. If we want to use the actual characters then we have to use single or double quotes.

Modifying Using the Form $’string’

[ahmed@amayem ~]$ IFS=$'n'; printf %q "$IFS"
$'n'

Looks good.

Modifying Using Single or Double Quotes

[ahmed@amayem ~]$ IFS="
> "; printf %q "$IFS"
$'n'

[ahmed@amayem ~]$ IFS='
> '; printf %q "$IFS"
$'n'