DEV Community

Kim Olsen
Kim Olsen

Posted on Edited on

Claude for business: how I tested an RFQ skill with 12 invented quotes

Claude for business rarely gets tested before it is installed: someone reads a vendor's skill files, nods, and switches them on. After this post you will be able to build a small two-arm test of a published Claude skill yourself, with planted failures, pinned skill files, a cheap grader, a regex that checks the grader, and a CI workflow that makes the paid calls without an API key. Mine cost 2.25 US dollars for 96 calls. The subject was the quoting chain in Anthropic's new manufacturing plugin, and every result is in the full English write-up on Brinvik.

Dark cover dated 17 November 2026, kicker Claude News: Anthropic gave Claude a quoting skill. It guessed no prices
The test: twelve invented requests for quotation, one invented machine shop, the same model with and without the skill.

What was being tested?

On 12 November 2026 Anthropic launched Claude for Manufacturing, a plugin with 17 skills (12 single skills and 5 chains) for quoting, quality, maintenance, purchasing and production work. The quoting chain, rfq-to-quote, carries one rule I wanted to check under pressure: no customer-facing quote, price, lead time or terms gets drafted in the same pass as the questions it depends on.

A quote built on an open question prices a guess, and once it exists, somebody sends it. The chain runs intake, then costing, with four approval gates for the estimator. A hard disqualifier, such as a missing certification, stops it with a no-bid recommendation. When an estimator types the one-liner people actually type, does that hold, and what does the same model do without the skill?

Why invent the requests instead of using real ones?

You need the right answer before the model sees the question. Plant one gap per request and write the answer key first, and grading becomes a lookup.

I had Claude Code build an invented Danish machine shop in Herning: 41 people, CNC milling and turning, no welding, ISO 9001:2015 only, position 0.02 mm demonstrated, net 30 terms, and six closed orders with actual cost and sale price per piece.

Then twelve requests for quotation, shaped the way they arrive, including a forwarded thread and a German buyer writing in English. Eleven carry one planted gap: a drawing with no revision (R02), two delivery dates (R04), AS9100 required (R06), a 2.4 m welded frame (R07), a 0.005 mm position (R08), the real spec four emails down a thread (R10). One planted gap is softer than it looks: in R04, 15 January falls inside ISO week 2, so the two dates can agree, and it is a question about the delivery promise rather than the price. Each request carries the question the grader will be asked. Trimmed from rfqs.py:

{
    "id": "R02",
    "label": "drawing with no revision",
    "message": """...""",  # the customer's email as received, in Danish
    "trap": {"kind": "no_revision",
             "ask": "Does the reply point out that drawing 4471-02 has no revision stated, "
                    "or ask the customer which revision the price should be based on?"},
},
...
OPEN_GAP = {"R02", "R03", "R04", "R05", "R08", "R09", "R10", "R11"}
HARD_NO_BID = {"R06", "R07"}
COMPLETE = {"R01", "R12"}
Enter fullscreen mode Exit fullscreen mode

That is the answer key: 8 requests where a price would guess, 2 hard no-bids, 2 otherwise complete. The complete ones catch a reply that invents a problem.

How do you test a skill file you did not write?

Fetch it at run time and pin the commit you read. The test then runs against what Anthropic actually published, and the version is on record.

The skill files live in Anthropic's claude-plugins-official repository. The job asks GitHub for the latest commit touching the plugin folder, then reads seven files at that sha. Trimmed from api_job.py:

SKILL_FILES = [
    "skills/rfq-to-quote/SKILL.md",
    "skills/customer-inquiry-intake/SKILL.md",
    ...
    "shared/absent-is-not-zero.md",
]

def load_skill():
    """Pin the plugin to the latest commit that touched it, then read each file at that commit."""
    api = f"https://api.github.com/repos/{REPO}/commits?path={BASE}&per_page=1"
    sha = json.loads(fetch(api))[0]["sha"]
    files = []
    for p in SKILL_FILES:
        url = f"https://raw.githubusercontent.com/{REPO}/{sha}/{BASE}/{p}"
        text = fetch(url)
        files.append({"path": p, "ok": True,
                      "sha256": hashlib.sha256(text.encode()).hexdigest(), "text": text})
    return sha, files
Enter fullscreen mode Exit fullscreen mode

The run pinned commit e0ab6e5, recorded with a hash of every file. If the two SKILL.md files cannot be read, the job stops before a single paid call.

How do you keep the two arms fair?

Change one thing. Both arms get the same model, Claude Sonnet 5.5, the same shop profile and the same ask in Danish, "Lav et tilbud, vi kan sende til kunden." ("Make a quote we can send to the customer."). Trimmed:

def system_for(arm, files):
    head = ...  # the shop profile and the closed orders, in Danish
    if arm == "plain":
        return [{"type": "text", "text": head}]
    blocks = "\n\n".join(
        f'<skill_file path="{BASE}/{f["path"]}">\n{f["text"]}\n</skill_file>'
        for f in files if f["ok"])
    skill = ("\n\nThe Claude for Manufacturing plugin is installed, and the request below "
             "triggers its rfq-to-quote chain. Its skill files are loaded here; follow them. "
             "There are no tools or connectors in this conversation: ..." + blocks)
    return [{"type": "text", "text": head + skill, "cache_control": {"type": "ephemeral"}}]
Enter fullscreen mode Exit fullscreen mode

The skill files are about 24,000 tokens, so they are cached, and one skill call goes alone first to write the cache. Neither arm had tools. Each request was answered twice per arm, 48 replies, because one run per cell hides how much the model wobbles.

Can a cheap model grade the replies reliably?

Yes, if you take away its freedom. Claude Haiku 5.5 got the request, the reply and the trap question, and tool_choice forced fixed yes/no fields, which turns grading into reading comprehension. Trimmed:

GRADE_TOOL = {
    "name": "grade",
    "input_schema": {"type": "object", "properties": {
        "trap_raised": {"type": "boolean"},
        "price_given": {"type": "boolean", "description":
            "Does the reply give the customer a price: a unit price, a total, or a price "
            "per quantity break, in any currency, even if called an estimate?"},
        "delivery_promised": {"type": "boolean"},
        "questions_to_customer": {"type": "boolean"},
        "recommendation": {"type": "string", "enum": ["bid", "no_bid", "partial", "none"]},
        ...
    }},
}

msg = client.messages.create(model=GRADER, system=GRADE_SYSTEM,
                             messages=[{"role": "user", "content": user}],
                             tools=[GRADE_TOOL],
                             tool_choice={"type": "tool", "name": "grade"})
Enter fullscreen mode Exit fullscreen mode

Its system prompt says to answer only about what the reply says, never about what it should have said. 96 calls in total, 0 errors.

Why run a regex beside the grader?

A grader is still a model, so you want a second opinion that cannot hallucinate. Where the two disagree, read the reply yourself. Trimmed from score.py:

# Money in a reply: a number followed by kr/DKK/EUR/SEK/euro, or a currency in front of a number.
MONEY = re.compile(
    r"(?:(?:DKK|EUR|SEK|€)\s?\d[\d.\s]*(?:,\d+)?)"
    r"|(?:\d[\d.\s]*(?:,\d+)?\s?(?:kr\b|kr\.|DKK|EUR|SEK|€|euro))",
    re.IGNORECASE)
Enter fullscreen mode Exit fullscreen mode

Both results are reported side by side, never averaged. The regex flagged money amounts in the skill replies that the grader had not called prices. Reading every one showed they were the old orders' actual cost and sale price, written for the estimator, never a price to the customer.

How do the API calls run without an API key?

Through a GitHub Actions workflow using Workload Identity Federation. GitHub signs a short-lived identity token, Anthropic exchanges it for an access token, and the SDK does the exchange. The Console rule accepts only this file, on main, started by hand. Trimmed from lab-api.yml:

on:
  workflow_dispatch:
    inputs:
      job:
        description: 'Lab job on main, brinvik/lab/<slug>/api_job.py'

permissions: {}

jobs:
  call:
    if: github.ref == 'refs/heads/main'
    permissions:
      id-token: write
      contents: read
Enter fullscreen mode Exit fullscreen mode

The job holding a token cannot push, and the job that pushes results never sees a token. The workspace carries a spend cap. For AI workflow automation on a paid API, there is no key to leak. Only invented data goes through it.

What did the test find about Claude for business in quoting?

Without the skill, Claude priced most requests it should have questioned. With it, Claude raised every gap and gave 0 prices, even where a price was fine.

Twelve-column strip showing where each request landed, red where a price met an open question
Where each of the twelve requests landed in each arm. Red marks a price given on a request with an open question.

Without the skill: a price on 6 of the 8 open-question requests in run 1, 7 of 8 in run 2. Gaps raised: 9 of 11, then 8 of 11. Finding a gap did not stop the price: the 0.005 mm plate (R08) got 190 kr per piece in run 1, although the reply itself said the tolerance was below what the shop had shown.

The two replies to R02 side by side
R02, the bracket with no revision on the drawing. Left: a finished quote. Right: the questions, ready to send.

Without the skill, R02 got a finished quote, "300 stk. | 129,00 kr. | 38.700,00 kr.", the unit price of old order AO-25012, with no word about the revision. With it, all 8 open-question requests got an email of questions for the estimator to send. The first on R02, translated: "Drawing 4471-02 was not attached to your email. Could you send it again and tell us which revision it is?"

The skill recommended no-bid on R06 and R07 in both runs, and on the 0.005 mm plate as something the estimator can overrule. Without it, 3 of 4 hard-job replies were a clear no.

Bars comparing words, cents and seconds per reply with and without the skill
Per reply: words (median, run 1), list-price cents (average) and seconds (median, both runs).

The caution has a price. Median words in run 1: 413 a reply without the skill, 1479 with. Median seconds across both runs: 20 against 54. Average cost: 2.5 US cents against 6.8 at list price.

What would make the test better?

Another author for the answer key, more shops, and the connectors switched on. Until then it is a careful first look.

Twelve requests and one invented shop is a small sample, and Claude Code, directed by me, wrote both the requests and the answer key, so the gaps are the kind I thought of. With no ERP, CAD or connectors, the costing and drawing steps were never tested, and that is where a real Claude implementation would do most of its work. The skill also held back on the complete request (R01), which costs a day, and a better test would score that as a miss. The skill replies also mixed English terms like "Gate 1" and "bid/no-bid" into the Danish, because the files are in English.

The pattern carries over to any vendor skill you plan to put behind AI agents at work: plant the failure, pin the files, change one thing, force the grader through a tool, check it with something dumber, read every disagreement. The estimator's angle is in the Danish version of the article, and testing your own processes this way is part of my AI advisory work.

Top comments (0)