Programmable tab completion with bash

We all use tab completion in the shell, but have you stopped to think about how it works? How with one command it will auto-complete only directory names, with another directories and filenames, but with another it will complete branch names for your VCS, for example? It's programmable! And so we can bend it to our will!

I recently had an annoyance. I use a tool called ts (task spooler) to run some CPU-intensive processes in the background, and to queue them so that only one such process runs at once. It is invoked like this:

$ ts command arg1 arg2 arg...

I wanted to be able to use tab completion for the command, and to then use that command's tab completion for subsequent arguments. After much swearing and cursing - the documentation in the bash manpage is not the best - and a bit of help from some nice people on Stack Overflow I came up with this:

complete -o bashdefault -o default -F __ts_bash_completions ts

__ts_bash_completions () {
    COMPREPLY=()

    if [ "$COMP_CWORD" -eq 1 ]; then
        COMPREPLY=($(compgen -c -- "${COMP_WORDS[COMP_CWORD]}"))
    else
        local command_completion_function="$(complete -p ${COMP_WORDS[1]} 2>/dev/null|sed 's/.*-F \([^ ]*\) .*/\1/')"

        if [ ! -z "$command_completion_function" ]; then
            COMP_CWORD=$(( COMP_CWORD - 1 ))
            COMP_LINE=$(echo $COMP_LINE|sed "s/^${COMP_WORDS[0]} //")
            COMP_WORDS=( "${COMP_WORDS[@]:1}" )

            $command_completion_function "${COMP_WORDS[0]}" "$2" "$3"
        fi
    fi
}

Let's go through it in detail. The first line tells the shell to use the function __ts_bash_completions when the user is typing the ts command and its subsequent arguments:

complete -F __ts_bash_completions ts

We then define that function.

__ts_bash_completions () {
    COMPREPLY=()

and the first thing we do is create an empty array COMPREPLY. bash completions populate this global variable to tell the shell what options are available. We then see how many complete words there are on the command line:

    if [ "$COMP_CWORD" -eq 1 ]; then

COMP_CWORD is another global variable that contains the number of complete words currently in the command. If that is 1 then the only word currently in the command is ts itself, we want to autocomplete the name of a command:

        COMPREPLY=($(compgen -c -- "${COMP_WORDS[COMP_CWORD]}"))

compgen (completion generator) generates a list of all the commands available in the $PATH which begin with ${COMP_WORDS[COMP_CWORD]}. This introduces yet another magic global variable, COMP_WORDS is an array (zero indexed) of all the words currently on the command line, including the one currently being typed, which may be empty. We pick the last one and pass that to compgen for it to use as a filter.

At this point we've tab completed the name of the command that ts is to run and deserve a beer.

Now we need to deal with tab completion of the arguments to the command that we want ts to run. First we need to find what function is used for completions for that command:

    else
        local command_completion_function="$(complete -p ${COMP_WORDS[1]} 2>/dev/null|sed 's/.*-F \([^ ]*\) .*/\1/')"

complete -p tells you exactly how tab completion is configured for a given command, and spits it out in the form of the command to use to configure it. The function will be the argument to the -F option, so we use a rather crude sed invocation to extract it, if it is present. The resulting string will be empty if no completion function is defined. We then check that it isn't empty and play around with the contents of the various COMP_* variables, setting them up to pretend that ts isn't involved:

        if [ ! -z "$command_completion_function" ]; then
            COMP_CWORD=$(( COMP_CWORD - 1 ))
            COMP_LINE=$(echo $COMP_LINE|sed "s/^${COMP_WORDS[0]} //")
            COMP_WORDS=( "${COMP_WORDS[@]:1}" )

We decrement COMP_CWORD because we want to ignore one of the completed words - that being ts itself. We remove ts from the start of COMP_LINE - which is a string containing the entire line of input. And we remove the first element from the COMP_WORDS array. Finally, we run the command's own completion function which will set COMPREPLY for us:

            $command_completion_function "${COMP_WORDS[0]}" "$2" "$3"

It appears that the calling convention for completion functions has changed over time. At some point in the past they were called with arguments, which only contain the command name and the last two arguments on the command line, then they were changed to instead use those global variables. In this case we can just use the arguments that were passed to our function altering only the first one, the name of the command whose arguments we are completing. Depending on how modern the completion function is for a given command, our ts completion needs to support both calling conventions.

There's one final wrinkle, which the observant amongst you may have noticed when I started going through this line by line. In my script I have -o bashdefault -o default when defining the relationship between ts and its completion function. Those are to cope with the case where the completion function leaves COMPREPLY empty. If that happens then -o bashdefault applies bash's own defaults for tab completion, ie helps you pick a filename, and if that doesn't return anything -o default applies readline's defaults. You can see this in action for git's completions. If you type git log <tab> in a git repo you get a list of branch names, and if you type git log foo<tab> you get a list of branches whose names begin with foo. But if there are no such branches you get a list of files whose names begin with foo, which is the shell's default action.

Anyway, after a great deal of wrestling with poor documentation, that's a little annoyance dealt with, and exactly the same code can also be used for other similar commands such as sudo and nohup. If you think you'll find it useful then the code is on Github and it's also in the ts mercurial repo so will no doubt be in a future release.