DEV Community

Cover image for One System Prompt, Five Text Jobs: Compliance Screening, Review Tagging and Contract Extraction from the Terminal
张洲诚(Zack.ZHANG)
张洲诚(Zack.ZHANG)

Posted on

One System Prompt, Five Text Jobs: Compliance Screening, Review Tagging and Contract Extraction from the Terminal

The backlog

A friend's small Etsy/Shopify shop got its product page rejected twice for over-claiming — and her keyword-blacklist checker had passed it both times, because those tools match strings, not meaning ("quietest" flags, "quiet-operation benchmark" sails through).

Fixing that led to counting everything else her shop publishes or receives and never reads: 300+ negative reviews sitting in the dashboard, hundreds of pre-sale messages, a 20-page supplier agreement. Pasting any of it into a chatbot produces prose — "customers seem unhappy with shipping" — which answers nothing you can act on. Which issue? How many? Enterprise review-analytics SaaS answers that for four figures a year, which a one-person shop won't pay.

Here's what worked instead: Alibaba Cloud Model Studio's CLI (bl), a system prompt that acts as a JSON schema, and a shell loop. Five kinds of text, one pattern. Show and tell below.

Messy text in, structured data out: the bl text chat pipeline

Setup

npm install -g bailian-cli
bl auth login
Enter fullscreen mode Exit fullscreen mode

Node 18+. API key from the Model Studio console — free tier included, and bl usage free shows what's left. Install guide here.

The one command everything else is a variation of

bl text chat --model qwen-turbo --quiet --system "You are a review analyzer. Output JSON only, no extra text. Fields: sentiment(positive/negative/neutral), dimension(quality/shipping/support/price/other), severity(high/medium/low), summary(under 12 words). If undeterminable, use 'unknown' — do not guess." --message "Stopped heating after three days and support took a week to reply."
Enter fullscreen mode Exit fullscreen mode
{"sentiment": "negative", "dimension": "quality", "severity": "high", "summary": "failed in three days, slow support"}
Enter fullscreen mode Exit fullscreen mode

Three schema rules I learned the hard way:

  • "JSON only, no extra text" — otherwise you get a polite "Sure! Here's the analysis:" prefix and your parser dies
  • Enumerate the allowed values — otherwise "shipping", "delivery" and "logistics" show up as three different dimensions and your group-by is confetti
  • "Use 'unknown', don't guess" — otherwise a review that just says "fine" gets a confidently invented label

The loop (this is the whole "batch framework")

mkdir -p out
while IFS= read -r line; do
  bl text chat --model qwen-turbo --quiet --system "You are a review analyzer. Output JSON only. Fields: sentiment, dimension(quality/shipping/support/price/other), severity(high/medium/low), summary(under 12 words). Use 'unknown' if undeterminable." --message "$line" >> out/results.jsonl
done < reviews.txt
Enter fullscreen mode Exit fullscreen mode

300 reviews, ~15 minutes, one JSON per line. Aggregation for small files: feed it back —

bl text chat --system "Input is line-delimited JSON of review analyses. Output a stats table: count by dimension descending, high-severity count per group, one-line summary of top three issues." --message "$(cat out/results.jsonl)"
Enter fullscreen mode Exit fullscreen mode

For thousands of rows, do the counting in jq or a pivot table instead — deterministic math shouldn't run on a probabilistic model.

The result that mattered

Shipping damage: 41 mentions. Product quality: 38. Support: 22. Her gut said quality was the top complaint; the data said shipping damage. She took the "41" to her carrier the next morning. The gap between gut and data is the product here.

Gut feeling vs the data: shipping damage was the real number one

Accuracy, honestly: 28/30 on a manual spot check. The two misses were sarcasm — every model's weak spot, not fixed by paying for a bigger one.

Same pattern, different system prompts

  • Ad compliance pre-check: JSON array of {quote, risk_type, explanation, suggested_rewrite} — caught an implied health claim ("say goodbye to lung-harming fumes") that no keyword blacklist would ever match. It's a self-check aid, not legal advice.
  • Contract clause extraction: {payment_terms, liability, dispute_resolution} with the load-bearing instruction "quote verbatim, never paraphrase" — paraphrased legal text is subtly wrong legal text.
  • Support inbox tagging: {intent, keyword} over a few hundred messages; compatibility questions turned out to be half of pre-sale volume, so the compatibility chart moved up the product page.
  • The inverse case — one long competitor analysis — flips the model strategy: default flagship, --max-tokens 8000 (default 4096 truncates long reports), and "never fabricate figures" in the system prompt.

Cost

bl usage stats --days 7 says the entire evening — 700+ calls — cost pocket change, mostly inside the free tier. The SaaS quote she'd gotten starts at four figures annually.

When not to do this

Forty reviews? Just read them. Can't write down what "high severity" means? The model can't tag it either. Anything with legal weight? This locates and drafts; a human decides.

If you've got a text backlog of your own, start with one review and one command: free tier here.

Curious: has anyone wired this kind of tagging loop into a scheduled job with drift monitoring? That's the part I haven't built yet.

Top comments (0)