Is it possible to categorize the bash autocomplete output?

Question

I have read the bash programmable completion on the GNU website, and many good Q&A on stackoverflow and unix.stackexchange.com. Fortunately, I have made an autocompletion script that works almost exactly as I want. However, there is a final touch I need with formatting the suggestions.

Is there a way to categorize the suggestions based on their types. For instance, I want something like this:

$ foo --bar # press TAB twice and get the result as below:
* Directories:
foo/   bar/   baz/
* Files:
foo.sh   bar.sh   baz.sh
* Options:
--foo=   --bar   --baz

instead of this:

$ foo --bar # press TAB twice and get the result as below:
--bar      --baz      --foo=      bar/      baz/
foo/       bar.sh     baz.sh      foo.sh

A minimal code snippet for the completion script would be:

_foo(){

   COMPREPLY=()
   local word="$2"
   local prev="$3"

   # Suggest directories
   COMPREPLY+=( $( compgen -d -- $word ) )
   
   # Suggest '*.sh' files with negating the '-X' filter pattern
   COMPREPLY+=( $( compgen -f -X "![.][sS][hH]$" -- $word ) )

   # Suggest options in the '-W' word list
   COMPREPLY+=( $( compgen -W "--foo= --bar --baz" -- $word ) )
}

complete -F _foo foo

I appreciate your time, thoughts, and insights :)

Update:

I tweaked @Socowi 's answer so that the extra \ won't show up before the = signs.

compgenSection() {
  title="* $1:"
  shift
  local entries
  mapfile -t entries < <(compgen "$@")
  (( "${#entries[@]}" == 0 )) && return
  COMPREPLY_SECTIONLESS+=("${entries[@]}")
  mapfile -tO "${#COMPREPLY[@]}" COMPREPLY < <(
    printf "%$((-COLUMNS/2))s\\n" "$title"
    printf %s\\n "${entries[@]}" | sort | column -s $'\n' | expand
  )
  [[ "${COMPREPLY[@]}" =~ "=" ]] && compopt -o nospace
}   
_foo(){
  local word="$2"
  COMPREPLY_SECTIONLESS=()
  compgenSection Directories -d -S / -- "$word"
  compgenSection Files -f -X '!*.[sS][hH]' -- "$word"
  compgenSection Options -W '--foo= --bar --baz' -- "$word"
  (( "${#COMPREPLY_SECTIONLESS[@]}" <= 1 )) &&
  COMPREPLY=("${COMPREPLY_SECTIONLESS[@]}")
}
complete -o nosort -F _foo foo

Answer 1

I cannot say for sure whether customizing the format is impossible, but if it is, here is a possible workaround just in case:

Add the section titles to COMPREPLY to show them at all. Now we have other problems to work around:

1. The completions are sorted. The section titles appear at the wrong location.
   Workaround: Use complete -o nosort ... to show the section titles and entries in the right order. You can manually sort the entries in each section again.
2. Since bash displays completions in multiple columns the section titles might appear in the same line as other entries.
   Workaround: Force each completion entry to be shown in its own line. Bash formats the entries in COMPREPLY into evenly sized columns. If one of the columns is wider than half of the terminal, then all entries will be displayed in their own line. Simply pad the section titles with spaces.
3. Now all completions are shown in single lines too, but we want them in columns.
   Workaround: Manually put all completions in columns. Then add each line (with possibly multiple entries) as a single entry in COMPGEN.
4. Pressing tab once to autocomplete doesn't work. Because of the section titles there is always more than one possibility.
   Workaround: make sure to adapt COMPREPLY accordingly when word=$2 changes. If there is only one match in all sections, then the section headers must be suppressed such that COMPREPLY contains only that single match. Here we also suppress empty sections to make it look a bit nicer.

Implementation

compgenSection() {
  title="* $1:"
  shift
  local entries
  mapfile -t entries < <(compgen "$@")
  (( "${#entries[@]}" == 0 )) && return
  COMPREPLY_SECTIONLESS+=("${entries[@]}")
  mapfile -tO "${#COMPREPLY[@]}" COMPREPLY < <(
    printf "%$((-COLUMNS/2))s\\n" "$title"
    printf %s\\n "${entries[@]}" | sort | column -s $'\n' | expand
  )
}   
_foo(){
  local word="$2"
  COMPREPLY_SECTIONLESS=()
  compgenSection Directories -d -- "$word"
  compgenSection Files -f -X '!*.[sS][hH]' -- "$word"
  compgenSection Options -W '--foo= --bar --baz' -- "$word"
  (( "${#COMPREPLY_SECTIONLESS[@]}" <= 1 )) &&
  COMPREPLY=("${COMPREPLY_SECTIONLESS[@]}")
}
complete -o nosort -o filenames -F _foo foo

Note that I had to change your compgen -f -X "![.][sS][hH]$" since the patterns accepted by compgen are globs and not regexes.

I also fixed a problem with array=( $(compgen ...) ) where filenames with spaces were splited into multiple completion entries. This still happens for filenames with linesbreaks in them. However, this seems like normal behavior: try touch $'b\na'; cat btabtab, which prints a b instead of something like $'b\na'.

I also add the option complete -o filenames such that entries with spaces in them are correctly quoted upon auto-completion. If this interferes with the completion for options (where you might want actual spaces) you can run the file and directory entries manually trough printf %q to still get correct quoting.

Testing

Here are some examples from an interactive session.
⌨️ is the prompt. ↹ marks where I pressed tab.

⌨️ mkdir /tmp/test/{,foo,bar,baz}; cd /tmp/test; touch foo.sh bar.sh baz.sh
⌨️ foo ↹↹
* Directories:                          
bar     baz     foo
* Files:                                
bar.sh  baz.sh  foo.sh
* Options:                              
--bar   --baz   --foo=
⌨️ foo f↹↹
* Directories:                          
foo
* Files:                                
foo.sh
⌨️ foo foo.↹ # auto-completes to `foo foo.sh`