Develop python CLI with subcommands using Click

While there are already great posts about developing CLI applications using Python's Click library showing examples of commands being setup from single python file, however you may instead want to organize commands into their own individual python files.

I had similar requirement while building a CLI application and spent quite some time in getting the structure I wanted to wire together and resolve minor errors. This tutorial is an attempt to show developing a successfully working example of CLI application with separate python files organized for each command.

For this tutorial, we will develop a simple CLI application that has multiple commands build and deploy like this:

$ python3 main.py
Usage: main.py [OPTIONS] COMMAND [ARGS]...

  CLI tool to manage full development cycle of projects

Options:
  --help  Show this message and exit.

Commands:
  build
  deploy

1. Create a new project directory with following structure and blank python files as below:
   

   $ tree
   .
   ├── commands
   │   ├── __init__.py
   │   ├── build.py
   │   ├── deploy.py
   └── main.py
   
   1 directory, 4 files
   

   You can create a separate python file inside commands dir for each command you want to expose to your main cli.

2. Install click python library using this:
   

   pip3 install click
   

3. Let's start with a starter code of main.py:
   

   import click
   
   @click.group(help="CLI tool to manage full development cycle of projects")
   def cli():
       pass
   
   if __name__ == '__main__':
       cli()
   

   📌 Note that we have annotated starter function cli() with @click.group indicating that this will be a group of commands.

4. Verify this part working successfully by running:
   

   $ python3 main.py
   Usage: main.py [OPTIONS] COMMAND [ARGS]...
   
     CLI tool to manage full development cycle of projects
   
   Options:
     --help  Show this message and exit.
   

5. Next, let's write code for build sub-command in commands/build.py:
   

   import click
   
   @click.command()
   @click.option('--docker', is_flag=True, help='Indicates the project should be built into docker image')
   def build(docker):
       if docker:
           print(f'Building this repo into a docker image...')
       else:
           print(f'Building this repo using default method...')
   

   📌 Note these points before moving forward:

   • We have defined a function build(docker) that is annotated with @click.command() to indicate that this is a command.
   • We also define a boolean type of option --docker for build command whose value gets passed to build() function. You can define additional options here for build command and write your own logic for this command here.

6. At this point, we have created a group (of commands) in main.py called "cli" and a separate command "build", but have not tied them together yet.

   Let's add this build command to our main group:
   
   6.1. Import commands/build library in main.py:
   

     from commands import build
   

   6.2. Add build command to the main group in main.py:
   

     cli.add_command(build.build)
   

   📌 Note:

   • We performed add_command() on cli object instead of click object
   • We added build.build to cli instead of build

7. Verify that you can now run build as a subcommand of your main program:
   

   $ python3 main.py build --help
   Usage: main.py build [OPTIONS]
   
   Options:
     --docker  Indicates the project should be built into docker image
     --help    Show this message and exit.
   

📌 If you have a working structure at this point, then you can easily follow similar process for adding new sub-commands into separate python files under commands/ dir or any other dir.

For example, let's add a deploy subcommand functionality to our main CLI program. This subcommand should expose 2 options (--env and --cloud) and prompt the user to enter input while displaying allowed values and a default value.

1. Define functionality and available options for deploy subcommand:
   

   import click
   
   @click.command()
   @click.option('--env', '-e', default="dev", type=click.Choice(['dev', 'stg', 'prd'], case_sensitive=False), prompt='Enter env name to deploy', help='Env to deploy')
   @click.option('--cloud', '-c', default="aws", type=click.Choice(['aws', 'gcp', 'azure'], case_sensitive=False), prompt='Enter cloud to deploy to', help='Cloud to deploy to')
   def deploy(env, cloud):
       print(f'Deploying current application artifact to {env} environment in {cloud} cloud...')
   

2. Import this new deploy subcommand to main CLI program:
   

   from commands import deploy
   

3. Add deploy subcommand to main CLI program:
   

   cli.add_command(deploy.deploy)
   

4. Verify that you can now run deploy subcommand:
   

   $ python3 main.py deploy
   Enter env name to deploy (dev, stg, prd) [dev]: 
   Enter cloud to deploy to (aws, gcp, azure) [aws]: 
   Deploying current application artifact to dev environment in aws cloud...
   

🏁 🏆 I hope you found this tutorial useful and it helped you implement a working CLI structure with commands organized into separate python files/dir. Feel free to leave questions or feedback in comments. 🔚

🛎 P.S.: Here are the final version of above files for reference:

main.py

import click

from commands import build
from commands import deploy


@click.group(help="CLI tool to manage full development cycle of projects")
def cli():
    pass


cli.add_command(build.build)
cli.add_command(deploy.deploy)

if __name__ == '__main__':
    cli()

commands/build.py

import click

@click.command()
@click.option('--docker', is_flag=True, help='Indicates the project should be built into docker image')
def build(docker):
    if docker:
        print(f'Building this repo into a docker image...')
    else:
        print(f'Building this repo using default method...')

commands/deploy.py

import click

@click.command()
@click.option('--env', '-e', default="dev", type=click.Choice(['dev', 'stg', 'prd'], case_sensitive=False), prompt='Enter env name to deploy', help='Env to deploy')
@click.option('--cloud', '-c', default="aws", type=click.Choice(['aws', 'gcp', 'azure'], case_sensitive=False), prompt='Enter cloud to deploy to', help='Cloud to deploy to')
def deploy(env, cloud):
    print(f'Deploying current application artifact to {env} environment in {cloud} cloud...')

Comments

[David SG]

Nice article! I had the same need years ago, but to avoid creating multiple CLIs for each project I've built aux4, a tool where you can define a JSON file with your commands and it generates documentation and it's also easy for sharing. Please check that out if you find interesting: aux4.io

[Drako]

Hey, just found your article, thanks for sharing! My CLI tool now is much more organized and clean!