Reliable shell quoting in shell

Sunday, 15 May 2005

If you need to generate shell commands with embedded filenames, you need a way to quote filenames safely. Otherwise, filenames containing characters that have special meaning in shell will, at best, cause your script to malfunction, or at worst, could be specifically constructed by an attacker to cause your script to execute arbitrary commands. The standard algorithm for quoting a string in shell is to enclose the string in single quotes. In order to avoid embdedded single quotes from prematurely terminating the quoting sequence, you replace them with '\''. In this way, the filename foo'bar yields the quoted string 'foo'\''bar'.

This is very easy to do in any language that supports some sort of search&replace operation, such as the s/// operator in sed or Perl. But suppose you do do not want to rely on another language, and you want to do this in pure shell – at least, in pure bash, since POSIX sh is not capable of any significant text manipulation.

Well, the replacement itself is not that hard to figure out: if $foo contains the text to be quoted, then echo ${foo//\'/\'\\\'\'} will produce a string with replaced single quotes. Of course, embedded whitespace is mangled, because I didn’t quote the entire thing. There is a reason for that: if you try echo "${foo//\'/\'\\\'\'}", you will find that there are extra backslashes in the output. But if you remove even one backslashe from the search&replace expansion, bash always thinks you forgot a quote or two! (I think this is a bug.)

It is fortunately possible to “cheat” to get around this: foo=$bar is actually the same as foo="$bar": both will properly preserve embedded whitespace. Therefore we can achieve the desired effect by way of something like ( bar=${foo/\'/\'\\\'\'} ; echo "$bar" )

And thus we get to the shell function I wrote to stash away this tidbit:

function shquot() {
  echo "'$quoted'"

You can use this for things such as echo "gunzip $( shquot "$infile" ) > $( shquot "$outfile" )" >> somescript and rest safe in the knowledge that it is not possible to construct filenames which will cause somescript to execute malicious code.