Today, I was curious to find out how many programs ending with
cat I had on my system. I remember using
zcat and wanted to know if there are similar programs. Pressing TAB after
cat only gives programs which start with
In my last post, I had mentioned that apropos is a way to search for what programs are available on your system. The search string for
apropos can be any regex. So,
apropos 'cat$' should solve the problem.
cat$ means all words ending with 'cat'
The output has
STAILQ_CONCAT (3) - implementations of singly-linked lists, singly-linked tail queues, lists and tail queues OPENSSL_strlcat (3ssl) - Memory allocation functions .. bzcat (1) - decompresses files to stdout cat (1) - concatenate files and print on the standard output fc-cat (1) - read font information cache files gencat (1) - Generate message catalog
Clearly, the top 2 do not look like programs. Why is
apropos then returning them?
Let's have a look at the
man apropos # apropos - search the manual page names and descriptions
apropos searches the
man pages. And looks like there are
man pages for other things and not just programs...
Digging deeper, let's try manual for the
man man # The table below shows the section numbers of the manual followed by the types of pages they contain. # 1 Executable programs or shell commands # 2 System calls (functions provided by the kernel) # 3 Library calls (functions within program libraries) # 4 Special files (usually found in /dev) # 5 File formats and conventions eg /etc/passwd # 6 Games # 7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7) # 8 System administration commands (usually only for root) # 9 Kernel routines [Non standard]
Ok. We are interested in are executable programs i.e section 1 of the man pages.
apropos has a way to limit which sections we search using
apropos -s 1 'cat$' gives us all programs ending with name
cat which have an entry in the man pages but it does not show us any programs which do not have a man page.
The way Bash knows which programs can be called directly by their name (e.g
ls) and not by their full path (e.g
/usr/bin/ls) is by looking at the
$PATH environment variable.
** Listing all executable files on path **
Here's a small bash snippet which lists the executable files in PATH (let's call it
#!/bin/bash # The directories in $PATH are separated by ":", so we split by it to get individual directories for pdir in $(echo "$PATH" | tr ":" "\n") do # We `find` all files in the directory which are executable and print the filename find "$pdir" -maxdepth 1 -executable -type f -printf "%f\n" done
If you prefer Python, here's a small Python program for the same (let's call it
from itertools import chain import os path_dirs = os.environ['PATH'].split(':') # Split PATH by ':' print(path_dirs) all_files = chain(*(os.walk(p) for p in path_dirs)) # Iterable of all files in the directories contained in PATH is_exec = lambda x : os.access(x, os.X_OK) # Function to check if a filename is executable execs = chain(*([f for f in fs if is_exec(os.path.join(r,f))] for r,_,fs in all_files)) for x in execs: print(x)
Running either our Bash or Python scripts will give us the correct output!
sh paths.sh | grep 'cat$' python3 paths.py | grep 'cat$'
When I press TAB TAB after typing a letter, I get a list of suggestions. How does Bash do that? The Bash manual says that it uses
compgen built-ins for suggesting completions.
compgen generates completions using a list of words (-W) or list of commands (-c). The latter is of particular interest to us.
compgen -c prints every executable on our path and all shell built-ins and shell-functions.
compgen --help prints following message:
compgen: compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W wordlist] [-F function] [-C command] [-X filterpat] [-P prefix] [-S suffix] [word]
Display possible completions depending on the options.
The options stand for:
- a : aliases
- b : shell builtins
- c : executable commands
- d : directories in current directory
- e : export variables
- f : files in current directory
- g : groups in system
- j : pending jobs (in background / stopped)
- k : Bash keywords
- s : System services
- u : users
- v : All shell variables
compgen -c | grep 'cat$' should give us every single executable ending with
Diving into this rabbit-hole has given me a better understanding of how Bash completion works, how apropos finds relevant programs and why man pages are organized into various sections.