Build a Model-Switchable Python CLI with an OpenAI-Compatible API

A small command-line tool is a useful way to test an LLM API before integrating it into a larger application. This tutorial builds a Python CLI that accepts a prompt, switches models, streams output, reports token usage, and handles common API errors.

The example uses HeFu's documented OpenAI-compatible Chat Completions endpoint.

Install the SDK

Create a virtual environment and install the OpenAI Python package:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade openai

On Windows PowerShell, activate it with:

.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade openai

Set the API key as an environment variable. On macOS or Linux:

export HEFU_API_KEY="your-api-key"
export HEFU_MODEL="deepseek-v4-flash"

On Windows PowerShell:

$env:HEFU_API_KEY="your-api-key"
$env:HEFU_MODEL="deepseek-v4-flash"

Build the CLI

Save this script as llm_cli.py:

import argparse
import os
import sys

from openai import (
    APIConnectionError,
    APIStatusError,
    APITimeoutError,
    OpenAI,
)


def main() -> int:
    parser = argparse.ArgumentParser(
        description="Send a prompt to a chat completion model."
    )
    parser.add_argument("prompt", help="Prompt to send to the model")
    parser.add_argument(
        "--model",
        default=os.getenv("HEFU_MODEL", "deepseek-v4-flash"),
        help="API model ID",
    )
    parser.add_argument(
        "--stream",
        action="store_true",
        help="Print text as it arrives",
    )
    args = parser.parse_args()

    api_key = os.getenv("HEFU_API_KEY")
    if not api_key:
        print("Missing HEFU_API_KEY.", file=sys.stderr)
        return 2

    client = OpenAI(
        api_key=api_key,
        base_url="https://hefu.hk/api/v1",
        timeout=30.0,
        max_retries=1,
    )

    messages = [
        {
            "role": "system",
            "content": "Answer clearly and state uncertainty when needed.",
        },
        {"role": "user", "content": args.prompt},
    ]

    try:
        if args.stream:
            stream = client.chat.completions.create(
                model=args.model,
                messages=messages,
                stream=True,
            )
            try:
                for chunk in stream:
                    if not chunk.choices:
                        continue

                    text = chunk.choices[0].delta.content or ""
                    print(text, end="", flush=True)
            finally:
                stream.close()

            print()
        else:
            response = client.chat.completions.create(
                model=args.model,
                messages=messages,
            )

            print(response.choices[0].message.content or "")

            if response.usage:
                print(
                    f"\nUsage: {response.usage.prompt_tokens} input tokens, "
                    f"{response.usage.completion_tokens} output tokens",
                    file=sys.stderr,
                )

    except APITimeoutError:
        print("Request timed out.", file=sys.stderr)
        return 1
    except APIConnectionError:
        print("Could not connect to the API.", file=sys.stderr)
        return 1
    except APIStatusError as exc:
        print(f"HTTP {exc.status_code}: {exc}", file=sys.stderr)
        return 1

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The configuration has three separate parts: base_url selects the API endpoint, api_key authenticates the request, and model selects the model for that call.

Run a request

For a complete response:

python llm_cli.py "Explain Python context managers in two paragraphs."

For incremental output:

python llm_cli.py --stream "Explain Python context managers in two paragraphs."

To switch models:

python llm_cli.py --model YOUR_MODEL_ID "Explain Python context managers."

Replace YOUR_MODEL_ID with an exact API ID from the current model catalog.

What changes when streaming is enabled?

A standard call returns a completed response. Its text is available in response.choices[0].message.content, and token usage may be available in response.usage.

A streaming call returns chunks. New text arrives in chunk.choices[0].delta.content. Some chunks contain no text, so the script checks choices and uses an empty string when content is missing.

This basic CLI reports usage only for non-streamed requests. If you need usage data from a stream, verify support for the relevant streaming options before relying on them.

Handle failures deliberately

The script handles timeouts, connection failures, and non-success HTTP responses separately. For an HTTP error, inspect the status code and response details: 401 suggests an authentication problem, 404 may indicate an incorrect route or model ID, and 429 indicates a rate limit.

A stream can fail after part of an answer has already appeared. Do not silently replay that request into the same output area. Mark it as incomplete and decide explicitly whether to retry.

Before using this pattern in an application

Keep API keys on the server, set appropriate timeouts, and avoid logging secrets or sensitive prompts. Test the exact features your application needs on each model. Sharing a Chat Completions request format does not guarantee identical model behavior or support for every optional parameter.

References: HeFu API documentation · OpenAI Python SDK

Disclosure: This article was drafted with AI assistance.