Rizwan Saleem

Posted on Jun 3

Building a resilient command-line tool with a plugin architecture in Python

#frontend #webdev

Building a resilient command-line tool with a plugin architecture in Python

Command-line tools are the backbone of many developers’ workflows. A robust CLI that’s easy to extend can save hours of toil and become a foundation for future tooling. In this tutorial, you’ll learn how to design and implement a resilient, extensible CLI in Python using a clean plugin architecture. You’ll see practical patterns for argument parsing, error handling, logging, configuration, and dynamic plugin loading, along with a complete example project you can adapt.

Outline

Why a plugin architecture matters for CLIs
Design goals and constraints
Core architecture overview
Step-by-step implementation
- Project scaffold
- Base CLI and command dispatch
- Configuration management
- Logging and error handling
- Plugin interface and discovery
- Example core commands
- Adding a third-party plugin
- Testing the CLI
- Packaging and distribution considerations
Practical tips and gotchas
Example repository structure

Why a plugin architecture matters

Extensibility: New features can be added without touching core code.
Isolation: Plugins can fail without crashing the entire tool if you catch errors properly.
Community growth: Users can contribute plugins that extend functionality in domain-specific ways.
Maintainability: Clear boundaries between core vs. extensions reduce coupling.

Design goals

Simplicity: Easy to understand, maintain, and extend.
Robustness: Clear error handling, meaningful messages, graceful shutdowns.
Deterministic behavior: Predictable command parsing and execution.
Discoverability: Plugins discovered at runtime with clear interfaces.
Testability: Unit tests for core, plugin contracts, and end-to-end flows.

Core architecture overview

Core: A minimal CLI that handles global options, configuration, and dispatching to commands.
Command model: Each command is an object with a run method, metadata, and its own argument parser.
Plugins: Implement a standardized interface and can register new commands; discovered via entry points or a plugins directory.
Plugin loading: Safe loading with fallbacks; isolated environments to avoid plugin-induced crashes.
Configuration: Centralized, hierarchical configuration (defaults, config file, environment variables, CLI overrides).
Logging: Configurable log level, structured logs for easier debugging.

Step-by-step implementation

1) Project scaffold
Create a new Python package, and a separate plugins directory for discovery.

Structure (high level)
- cli_tool/
- init.py
- core.py
- config.py
- logger.py
- commands/
  - init.py
  - base_command.py
  - whoami.py
  - echo.py
- plugins/
  - init.py
- plugins_example/
- plugin_sample.py
- setup.py or pyproject.toml
- tests/
- test_core.py
- test_plugins.py

2) Core CLI and command dispatch
Implement a minimal dispatcher that loads available commands, including plugins.

core.py key ideas:
- Use argparse for global options (config file, verbose, version).
- A registry mapping command names to command classes.
- A load_plugins function to discover and register plugin commands.
- A main() function that parses args, loads config, and executes the selected command.

Example (core.py):
from future import annotations

import argparse
import sys
from typing import Dict, Type

from .config import load_config
from .logger import get_logger
from .commands.base_command import BaseCommand

COMMAND_REGISTRY: Dict[str, Type[BaseCommand]] = {}

def register_command(name: str, cmd_cls: Type[BaseCommand]) -> None:
COMMAND_REGISTRY[name] = cmd_cls

def load_plugins() -> None:
# Simple dynamic discovery: import all modules from plugins package
try:
import plugins as _plugins
for attr in dir(_plugins):
obj = getattr(_plugins, attr)
if isinstance(obj, type) and issubclass(obj, BaseCommand) and obj is not BaseCommand:
register_command(obj.name, obj)
get_logger().debug("Plugins loaded: %s", list(COMMAND_REGISTRY.keys()))
except Exception as e:
get_logger().warning("Plugin loading failed: %s", e)

def main(argv=None) -> int:
if argv is None:
argv = sys.argv[1:]

parser = argparse.ArgumentParser(prog="cli-tool", description="A resilient CLI with plugin support")
parser.add_argument("version", action="store_true", help="Show version and exit")
parser.add_argument("config", help="Path to config file")
parser.add_argument("-v", "verbose", action="store_true", help="Increase verbosity")

# First pass to handle global flags
ns, rest = parser.parse_known_args(argv)

# Initialize logging based on verbosity
logger = get_logger(ns.verbose)

# Load configuration
config = load_config(ns.config)

# Load core commands and plugins
load_plugins()
# Core commands could register here as well
if ns.version:
    print("cli-tool version 0.1.0")
    return 0

if not rest:
    parser.print_help()
    return 0

cmd_name = rest
cmd_args = rest[1:]

if cmd_name not in COMMAND_REGISTRY:
    logger.error("Unknown command: %s", cmd_name)
    parser.print_help()
    return 2

cmd_cls = COMMAND_REGISTRY[cmd_name]
command = cmd_cls(config=config, logger=logger)
return command.run(cmd_args)

if name == "main":
raise SystemExit(main())

3) Configuration management
A simple config system with defaults, environment overrides, and a config file.

config.py:
import os
from typing import Any, Dict

DEFAULT_CONFIG = {
"log_level": "INFO",
"data_dir": "~/.cli_tool",
"name": "cli-tool",
}

def _expand_paths(cfg: Dict[str, Any]) -> Dict[str, Any]:
if "data_dir" in cfg:
cfg["data_dir"] = os.path.expanduser(cfg["data_dir"])
return cfg

def load_config(config_path: str | None) -> Dict[str, Any]:
cfg = dict(DEFAULT_CONFIG)
if config_path:
try:
with open(config_path, "r") as f:
for line in f:
line = line.strip()
if not line or line.startswith("#"):
continue
k, , v = line.partition("=")
cfg[k.strip()] = v.strip()
except FileNotFoundError:
pass
# Environment variable overrides
for key in cfg.keys():
env_key = f"CLI_TOOL{key.upper()}"
if env_key in os.environ:
cfg[key] = os.environ[env_key]
return _expand_paths(cfg)

4) Logging and error handling
A small logger wrapper to standardize log messages.

logger.py:
import logging

_LOGGER: dict[str, logging.Logger] = {}

def get_logger(level: bool = False) -> logging.Logger:
key = "default"
if key in _LOGGER:
logger = _LOGGER[key]
else:
logger = logging.getLogger("cli-tool")
handler = logging.StreamHandler()
formatter = logging.Formatter("%(levelname)s: %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.propagate = False
_LOGGER[key] = logger
logger.setLevel(logging.DEBUG if level else logging.INFO)
return logger

5) Plugin interface and discovery
BaseCommand defines the contract. Plugins implement new commands by subclassing BaseCommand and registering themselves.

commands/base_command.py:
from future import annotations
from abc import ABC, abstractmethod
from typing import Any, Dict
from . import all as _unused # placeholder for package init

class BaseCommand(ABC):
name: str = "base"

def __init__(self, config: Dict[str, Any], logger) -> None:
    self.config = config
    self.logger = logger

@abstractmethod
def add_arguments(self, parser: "argparse.ArgumentParser") -> None:
    pass

@abstractmethod
def run(self, args: list[str]) -> int:
    pass

whoami.py:
import argparse
from .base_command import BaseCommand

class WhoAmICommand(BaseCommand):
name = "whoami"

def add_arguments(self, parser: "argparse.ArgumentParser") -> None:
    parser.add_argument("short", action="store_true", help="Short output")

def run(self, args: list[str]) -> int:
    parser = argparse.ArgumentParser(prog="cli-tool whoami")
    self.add_arguments(parser)
    ns = parser.parse_args(args)
    if ns.short:
        print("user@example.com")  # placeholder
    else:
        print("CLI Tool User: user@example.com")
    return 0

echo.py:
import argparse
from .base_command import BaseCommand

class EchoCommand(BaseCommand):
name = "echo"

def add_arguments(self, parser: "argparse.ArgumentParser") -> None:
    parser.add_argument("message", nargs="?", default="hello")

def run(self, args: list[str]) -> int:
    parser = argparse.ArgumentParser(prog="cli-tool echo")
    self.add_arguments(parser)
    ns = parser.parse_args(args)
    print(ns.message)
    return 0

init.py in commands to export base structures
from .base_command import BaseCommand
from .whoami import WhoAmICommand
from .echo import EchoCommand

6) Example core commands
Register core commands on startup.

In core.py, modify to register built-ins:
def register_builtins():
register_command(WhoAmICommand.name, WhoAmICommand)
register_command(EchoCommand.name, EchoCommand)

In main(), after loading plugins, call register_builtins() before dispatch.

7) Adding a third-party plugin
Create a separate package or module that the loader discovers.

plugins/plugin_sample.py:
from .base_command import BaseCommand
from typing import Dict

class DatePlugin(BaseCommand):
name = "date"

def add_arguments(self, parser):
    parser.add_argument("format", default="%Y-%m-%d")

def run(self, args):
    import datetime
    parser = self._build_parser()  # or replicate pattern
    ns = parser.parse_args(args)
    print(datetime.datetime.now().strftime(ns.format))
    return 0

Note: Proper plugin infrastructure would require a consistent way to access the parent command’s parser. The example keeps the concept simple; adapt as needed.

8) Testing the CLI

Test core dispatch: simulate argv, ensure correct command runs.
Test plugin loading: create a temporary plugins package with a test command and ensure it appears in registry.
Test error paths: unknown command, missing args.

Test snippet (pytest style):
def test_echo_command_runs(monkeypatch, capsys):
from cli_tool.core import main
exit_code = main(["echo", "hi"])
assert exit_code == 0
captured = capsys.readouterr()
assert "hi" in captured.out

9) Packaging and distribution considerations

Use a plugin loading mechanism compatible with environments (editable installs, plugin directories, or setuptools entry points).
If using entry points, declare in setup.py: entry_points={ "cli_tool.plugins": [ "date = plugins.plugin_sample:DatePlugin", ] }
At runtime, importlib.metadata.entry_points(group="cli_tool.plugins") to load plugins.

Practical tips and gotchas

Isolation: Consider running plugins in a restricted subprocess or with a safe sandbox if plugins execute untrusted code.
Versioning: Pin plugin API to a stable contract; document required methods (add_arguments, run) and expected behaviors.
Config precedence: Decide whether CLI flags override config file, which overrides environment, and document the order clearly.
Help text: Keep command-specific help concise and automatically reflect added arguments from add_arguments.
Local development: For rapid iteration, implement a lightweight in-repo plugin that’s easy to enable/disable via environment variable.

Example repository structure (concrete starting point)
cli-tool/

init.py
core.py
config.py
logger.py
commands/
- init.py
- base_command.py
- whoami.py
- echo.py
plugins/
- init.py
tests/
- test_core.py
- test_plugins.py

How to adapt this into a real project

Start small: ensure the core can run a couple of built-in commands reliably before adding plugins.
Incremental plugin loading: begin with a static registry of built-in plugins, then introduce dynamic loading via the plugins package or entry points.
Documentation: maintain a CONTRIBUTING.md with plugin API guidelines, including examples.
CI: run tests on PRs to guard against plugin breakages.

Illustrative example: a minimal working plugin

Core registers built-ins: echo and whoami.
Plugin defines a Date plugin that prints the current date.
Running: python -m cli_tool date format "%A, %d %B %Y"
Expected output: "Wednesday, 03 June 2026" (example date)

Would you like this tutorial tailored to a specific Python version (e.g., 3.11 vs 3.12), or adjusted for a particular packaging approach (pip, poetry, or a monorepo layout)? I can also provide a ready-to-run GitHub repo template with all files wired up and a minimal CI workflow.

Rizwan Saleem | https://rizwansaleem.co