DEV Community: HeFuAI

Build a Model-Switchable Python CLI with an OpenAI-Compatible API

HeFuAI — Fri, 30 Oct 2026 06:27:01 +0000

A small command-line tool is a useful way to test an LLM API before integrating it into a larger application. This tutorial builds a Python CLI that accepts a prompt, switches models, streams output, reports token usage, and handles common API errors.

The example uses HeFu's documented OpenAI-compatible Chat Completions endpoint.

Install the SDK

Create a virtual environment and install the OpenAI Python package:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade openai

On Windows PowerShell, activate it with:

.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade openai

Set the API key as an environment variable. On macOS or Linux:

export HEFU_API_KEY="your-api-key"
export HEFU_MODEL="deepseek-v4-flash"

On Windows PowerShell:

$env:HEFU_API_KEY="your-api-key"
$env:HEFU_MODEL="deepseek-v4-flash"

Build the CLI

Save this script as llm_cli.py:

import argparse
import os
import sys

from openai import (
    APIConnectionError,
    APIStatusError,
    APITimeoutError,
    OpenAI,
)


def main() -> int:
    parser = argparse.ArgumentParser(
        description="Send a prompt to a chat completion model."
    )
    parser.add_argument("prompt", help="Prompt to send to the model")
    parser.add_argument(
        "--model",
        default=os.getenv("HEFU_MODEL", "deepseek-v4-flash"),
        help="API model ID",
    )
    parser.add_argument(
        "--stream",
        action="store_true",
        help="Print text as it arrives",
    )
    args = parser.parse_args()

    api_key = os.getenv("HEFU_API_KEY")
    if not api_key:
        print("Missing HEFU_API_KEY.", file=sys.stderr)
        return 2

    client = OpenAI(
        api_key=api_key,
        base_url="https://hefu.hk/api/v1",
        timeout=30.0,
        max_retries=1,
    )

    messages = [
        {
            "role": "system",
            "content": "Answer clearly and state uncertainty when needed.",
        },
        {"role": "user", "content": args.prompt},
    ]

    try:
        if args.stream:
            stream = client.chat.completions.create(
                model=args.model,
                messages=messages,
                stream=True,
            )
            try:
                for chunk in stream:
                    if not chunk.choices:
                        continue

                    text = chunk.choices[0].delta.content or ""
                    print(text, end="", flush=True)
            finally:
                stream.close()

            print()
        else:
            response = client.chat.completions.create(
                model=args.model,
                messages=messages,
            )

            print(response.choices[0].message.content or "")

            if response.usage:
                print(
                    f"\nUsage: {response.usage.prompt_tokens} input tokens, "
                    f"{response.usage.completion_tokens} output tokens",
                    file=sys.stderr,
                )

    except APITimeoutError:
        print("Request timed out.", file=sys.stderr)
        return 1
    except APIConnectionError:
        print("Could not connect to the API.", file=sys.stderr)
        return 1
    except APIStatusError as exc:
        print(f"HTTP {exc.status_code}: {exc}", file=sys.stderr)
        return 1

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

The configuration has three separate parts: base_url selects the API endpoint, api_key authenticates the request, and model selects the model for that call.

Run a request

For a complete response:

python llm_cli.py "Explain Python context managers in two paragraphs."

For incremental output:

python llm_cli.py --stream "Explain Python context managers in two paragraphs."

To switch models:

python llm_cli.py --model YOUR_MODEL_ID "Explain Python context managers."

Replace YOUR_MODEL_ID with an exact API ID from the current model catalog.

What changes when streaming is enabled?

A standard call returns a completed response. Its text is available in response.choices[0].message.content, and token usage may be available in response.usage.

A streaming call returns chunks. New text arrives in chunk.choices[0].delta.content. Some chunks contain no text, so the script checks choices and uses an empty string when content is missing.

This basic CLI reports usage only for non-streamed requests. If you need usage data from a stream, verify support for the relevant streaming options before relying on them.

Handle failures deliberately

The script handles timeouts, connection failures, and non-success HTTP responses separately. For an HTTP error, inspect the status code and response details: 401 suggests an authentication problem, 404 may indicate an incorrect route or model ID, and 429 indicates a rate limit.

A stream can fail after part of an answer has already appeared. Do not silently replay that request into the same output area. Mark it as incomplete and decide explicitly whether to retry.

Before using this pattern in an application

Keep API keys on the server, set appropriate timeouts, and avoid logging secrets or sensitive prompts. Test the exact features your application needs on each model. Sharing a Chat Completions request format does not guarantee identical model behavior or support for every optional parameter.

References: HeFu API documentation · OpenAI Python SDK

Disclosure: This article was drafted with AI assistance.

Calling DeepSeek, Kimi, and Qwen From Outside China: The Real Blockers (and How to Check You're Not Being Marked Up)

HeFuAI — Wed, 28 Oct 2026 02:05:08 +0000

If you've tried to call DeepSeek, Kimi, or Qwen from outside mainland China, you already know the model isn't the problem. The weights are fine. The context window is fine. What stops people is the account layer: identity, payment rails, and procurement.

Most posts on this topic are either stale (written before the 2025–2026 access changes) or thin affiliate content that ends with "just use my gateway." This one tries to be the post I wish I'd had: what actually blocks you, which paths exist today, and — the part nobody writes — how to verify that a gateway isn't quietly marking up the upstream price.

Disclosure up front: We build one of these gateways (hefu.hk). We've tried to keep the checklist below one we'd want applied to us too. If it reads like a sales pitch, that's a failure on my part.

The three blockers (they're not what you think)

Identity. Several first-party consoles are built around a mainland mobile number (+86) and SMS verification. Foreign numbers often don't receive the code.
Payment. Funding an account frequently runs through domestic rails — Alipay or WeChat Pay tied to a Chinese bank account. A US/EU card may or may not work depending on the vendor and your region.
Procurement. Even when a developer gets a key working, finance asks for an invoice with a legal entity, a PO number, or a receipt they can reimburse. A personal Alipay top-up doesn't satisfy that.

Only the third one is usually fatal for a company. The first two are annoying; the third one sends you back to a Western model at 10x the price.

Path 1 — First-party international endpoints
This is the option most people miss: several Chinese labs now run international consoles separate from their mainland platforms.

DeepSeek — platform.deepseek.com, OpenAI-compatible at https://api.deepseek.com
Moonshot / Kimi — platform.moonshot.ai (international, USD) vs. platform.moonshot.cn (mainland, RMB)
Qwen / Alibaba — Model Studio international, e.g. https://dashscope-intl.aliyuncs.com/compatible-mode/v1 (Singapore) and a US region
Zhipu / GLM — api.z.ai for the international surface

Because they speak the OpenAI shape, the migration is usually three values:
from openai import OpenAI

client = OpenAI(
api_key="YOUR_KEY",
base_url="https://api.deepseek.com", # or dashscope-intl, api.z.ai
)

resp = client.chat.completions.create(
model="deepseek-chat",
messages=[{"role": "user", "content": "Explain MoE in one paragraph."}],
max_tokens=300,
)
print(resp.choices[0].message.content)

The honest caveat: availability, card acceptance, and even whether a +86 number is required vary by vendor, region, and month. Check the vendor's live pricing and signup page before you assume a path is open or closed. Don't trust a blog post (including this one) as your source of truth for today's signup flow.

When this path wins: you want a first-party relationship, the latest model tier, and you're willing to manage one dashboard per vendor.

When it loses: three vendors means three keys, three invoices, three rate-limit policies, and three support queues.

Path 2 — Self-host the open weights
DeepSeek, Qwen, and several GLM variants publish open weights. If data cannot leave your VPC — finance, legal, regulated manufacturing — this is often the only acceptable answer.

pip install vllm
vllm serve Qwen/Qwen3-235B-A22B --tensor-parallel-size 4

The catch is that you now own the serving stack: GPU budget, quantization choices, cold starts, autoscaling, and every future model upgrade. For a two-person team trying to ship a feature this week, that's a full-time job you didn't sign up for.

Rule of thumb: self-host for data-control reasons, not to save money. At low volume you'll usually lose on idle GPU time.

Path 3 — A unified gateway
One key, one endpoint, one invoice, many models. The appeal is obvious, and it's why this space has a dozen providers.

The risk is equally obvious: you're trusting a middleman with your traffic and your billing, and the middleman's margin is often invisible. A gateway's price is upstream cost × margin. When the upstream price moves (and Chinese providers change prices often, including peak/off-peak windows), a non-transparent gateway can quietly widen that margin.

That's the real question worth asking — not "who is 4% cheaper this week," but "can I verify what I'm paying for."

The checklist: 7 questions before you route production traffic
Ask any gateway these. A good one answers all seven without flinching.

Price parity, same day. Open the vendor's public pricing page next to the gateway's page and compare per-1M input/output. If it's higher, ask why and by how much.
Top-up fees and FX spread. Is there a deposit fee? A percentage on top-ups? A currency conversion margin? A "credits" system that hides the rate?
Token-level billing. Are input, output, and cached input billed separately and shown in the usage log? Cached input on DeepSeek is dramatically cheaper — if the gateway doesn't pass that through, you're overpaying on long system prompts.
Upstream transparency. Does the dashboard tell you which provider and model version served the request? Or just a marketing name?
Invoices and sub-accounts. Can you get a proper receipt / invoice for finance? Can you issue scoped keys per team member or per environment?
Data handling. Do they retain prompts? Is there PII redaction? What's the retention window? Get this in writing if you're in a regulated domain.
Failure behavior. What happens on upstream 429/5xx — failover to another provider, or a raw error to your users? Ask for the actual routing policy.
If a provider dodges #1 or #2, that's your answer.

A production-shaped client (framework-agnostic)
Whichever path you pick, wrap it once so you can swap providers later:

import os, time
from openai import OpenAI, APIError, RateLimitError

client = OpenAI(
api_key=os.environ["LLM_API_KEY"],
base_url=os.environ["LLM_BASE_URL"],
)

def ask(prompt: str, model: str, retries: int = 3):
for attempt in range(retries):
try:
stream = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
yield delta
return
except (RateLimitError, APIError):
if attempt == retries - 1:
raise
time.sleep(2 ** attempt)

Two details worth keeping: retry rate limits and 5xx, never retry a 400; and always cap max_tokens, because output tokens are where the bill hides.

How I'd choose
Need it working this week, one vendor → first-party international endpoint.
Data can't leave your VPC → self-host the open weights.
Multiple models, one bill, small team → gateway, but run the checklist first.
Company procurement involved → decide on invoices and sub-accounts before you decide on price.
The models are genuinely good and genuinely cheaper. The friction was never the model — it was the account layer. Pick the path that matches your constraint, and make the billing transparent before you scale traffic.

Disclosure: We build hefu.hk, an OpenAI-compatible gateway for Claude/GPT/Gemini plus Chinese models. We price at official rates with no top-up fee and provide token-level billing, receipts, and sub-accounts — which is exactly the checklist above. If you evaluate us, use the same seven questions.