Getting compgen to include slashes on directories when looking for files

Question

I'd like to get the following behavior from my custom completion

Given

$ mkdir foo
$ touch foo faz/bar faz/baz

I'd like to get this

$ foo -u <tab><tab> =>
foo faz/

$ foo -u fa<tab><tab> =>
foo -u faz/

$ foo -u faz/<tab><tab> =>
bar baz

I assumed that compgen -f f would output foo faz/, but it outputs foo faz which doesn't help me much.

Do I need to post-process the output or is there some magic combination of options to compgen that work?

Answer 1

I ran into the same problem. Here's the workaround I'm using:

1. Register the completion function with -o default, e.g., complete -o default -F _my_completion.
2. When you want to complete a filename, just set COMPREPLY=() and let Readline take over (that's what the -o default does).

There's a potential problem with this -- you might be using COMPREPLY=() to deny completion where it's not appropriate, and now that won't work anymore. In Bash 4.0 and above, you can work around this using compopt as follows:

1. At the top of your completion function, always run compopt +o default. This disables Readline filename completion when COMPREPLY is empty.
2. When you want to complete a filename, use compopt -o default; COMPREPLY=(). In other words, enable Readline filename completion only when you need it.

I haven't figured out a full workaround for pre-4.0 Bash, but I have something that works well enough for me. I can describe this if anyone really cares, but hopefully these older Bash versions will soon fall out of common use anyway.

Answer 2

Previous answers require bash 4, or aren't composable with other compgen-based completions for the same command. The following solution doesn't require compopt (so it works with bash 3.2) and it is composable (for example you can easily add additional filtering to only match certain file extensions). Skip to the bottom if impatient.

---

You can test compgen by running it directly on the command line:

compgen -f -- $cur

where $cur is the word you would've typed so far during interactive tab-completion.

If I'm in a directory that has a file afile.py and a sub-directory adir, and cur=a then the above command shows the following:

$ cur=a
$ compgen -f -- $cur
adir
afile

Note that compgen -f shows files and directories. compgen -d shows only directories:

$ compgen -d -- $cur
adir

Adding -S / will add a trailing slash to every result:

$ compgen -d -S / -- $cur
adir/

Now, we can try listing all files and all directories:

$ compgen -f -- $cur; compgen -d -S / -- $cur
adir
afile.py
adir/

Note that there's nothing stopping you from calling compgen more than once! In your completion script you would use it like this:

cur=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=( $(compgen -f -- "$cur"; compgen -d -S / -- "$cur") )

Unfortunately, this still doesn't give us the behaviour that we want, because if you type ad<TAB> you have two possible completions: adir and adir/. So ad<TAB> will complete to adir, at which point you still have to type in the / to disambiguate.

What we need now is a function that will return all files but no directories. Here it is:

$ grep -v -F -f <(compgen -d -P '^' -S '$' -- "$cur") \
>     <(compgen -f -P '^' -S '$' -- "$cur") |
>     sed -e 's/^\^//' -e 's/\$$//'
afile.py

Let's break it down:

• grep -f file1 file2 means show the lines in file2 that match any of the patterns in file1.
• -F means the patterns in file1 must match exactly (as a substring); they aren't regular expressions.
• -v means invert the match: Only show lines that aren't in file1.
• <(...) is bash process substitution. It allows you to run any command in the places where a file is expected.

So we're telling grep: here's a list of files -- remove any that match this list of directories.

$ grep -v -F -f <(compgen -d -P '^' -S '$' -- "$cur") \
>     <(compgen -f -P '^' -S '$' -- "$cur")
^afile.py$

I've added beginning and end markers with compgen's -P ^ and -S '$' because grep's -F does substring matching and we don't want to remove a filename like a-file-with-adir-in-the-middle just because its middle part matched a directory's name. Once we have the list of files we remove those markers with sed.

Now we can write a function that does what we want:

# Returns filenames and directories, appending a slash to directory names.
_mycmd_compgen_filenames() {
    local cur="$1"

    # Files, excluding directories:
    grep -v -F -f <(compgen -d -P ^ -S '$' -- "$cur") \
        <(compgen -f -P ^ -S '$' -- "$cur") |
        sed -e 's/^\^//' -e 's/\$$/ /'

    # Directories:
    compgen -d -S / -- "$cur"
}

You use it like this:

_mycmd_complete() {
    local cur=${COMP_WORDS[COMP_CWORD]}
    COMPREPLY=( $(_mycmd_compgen_filenames "$cur") )
}
complete -o nospace -F _mycmd_complete mycmd

Note that -o nospace means you don't get a space after a directory's /. For normal files we added a space at the end with sed.

One nice thing about having this in a separate function is that it's easy to test! For example here's an automated test:

diff -u <(printf "afile.py \nadir/\n") <(_mycmd_compgen_filenames "a") \
    || { echo "error: unexpected completions"; exit 1; }

Answer 3

This behaviour is influenced by the readline variable mark-directories (and the related mark-symlinked-directories). I believe the variable should be set on for complete to print a trailing slash on directory names (the default in bash v3.00.16). Seemingly the related behaviour of compgen doesn't append a slash to directory names :-\

Set the value of mark-directories alternately to on and off then retry your test:-

bind 'set mark-directories on'
bind 'set mark-directories off'

To make the change permanent for future invocations of bash, add the following to your INPUTRC file, commonly ~/.inputrc:-

$if Bash
# append '/' to dirnames
set mark-directories on
$endif

The tip to set readline variables in the current shell was found here: https://unix.stackexchange.com/a/27545. I didn't determine how to test the current value of a readline variable.

Additional Ideas

Perhaps only academic interest...

Create a list of directory names only and append a slash:-

compgen -d -S / f

Answer 4

Use the -o filenames option passed to the complete directive. As long as the contents of COMPREPLY are valid paths the slash will be appended as appropriate while still adding a space after completion of non-directory matches. (Works on bash 3.2.)

_cmd() {
  COMPREPLY=( $( compgen -f -- "$cur" ))
  COMPREPLY=( $( compgen -A file -- "$cur" ))  # same thing
}

complete -o filenames -F _cmd cmd

See info -n "(bash)Programmable Completion Builtins":

complete

... -o COMP-OPTION ...

     filenames
           Tell Readline that the compspec generates filenames, so
           it can perform any filename-specific processing (like
           adding a slash to directory names or suppressing
           trailing spaces).  This option is intended to be used
           with shell functions specified with `-F'.

And:

-A ACTION

... 'file'

           File names.  May also be specified as -f.

Note: compgen and complete have the same arguments.