Cut Claude API Costs 80% by Splitting Vision and Reasoning Tasks

yogeshchavan2008 — Thu, 09 Jul 2026 12:11:04 +0000

The Problem That Forced the Question

I was building an ingestion pipeline for a personal knowledge system—drop in a PDF, get structured OKF-format concept articles out the other side. Anthropic's vision models are genuinely good at reading image-only PDFs (scanned docs, Medium exports printed to PDF), so I wired them into my OCR layer.

Then I looked at the cost estimate for my sample: a 26-page image-only PDF. Running everything through claude-opus-4-8 would cost $1.00–$1.30 for a single document. Scale that to a few dozen docs and suddenly my personal knowledge base has a real operational cost.

The fix was obvious once I thought about it—but I hadn't thought about it.

The Insight: OCR Is Transcription, Not Reasoning

Vision OCR and concept extraction are fundamentally different tasks:

OCR: "What text is on this page?" Rote transcription. High-contrast text on a white background. No ambiguity. This does not need Opus.
Extraction: "Given this 15,000-character document about concurrency, identify the atomic concepts, their prerequisites, define their difficulty levels, and generate interview questions." This benefits from a stronger model.

Once I separated these mentally, the routing became obvious.

The Pricing Math

Current Claude API pricing (per 1M tokens):

Model	Input	Output
`claude-opus-4-8`	$5.00	$25.00
`claude-sonnet-4-6`	$3.00	$15.00
`claude-haiku-4-5`	$1.00	$5.00

Image tokens are roughly (width × height) / 750. My PDF rendered at 200 DPI produces Letter-size pages at ~1700×2200px—about 3.7MP—which hits near Opus's 4,800-token ceiling per page.

26 pages × ~4,800 image tokens per page = ~125,000 image tokens just for OCR.

Route	Est. cost	Notes
Opus for everything	~$1.10	wasteful
Haiku OCR + Opus extraction	~$0.20	OCR is cheap on Haiku
Haiku OCR + Sonnet extraction	~$0.18	sweet spot
Haiku for both	~$0.12	lower extraction quality

That's an 84% reduction by using Haiku for the vision calls and Sonnet for the one extraction call.

What This Looks Like in Code

In .env:

OCR_ENGINE=claude
CLAUDE_VISION_MODEL=claude-haiku-4-5
CLAUDE_MODEL=claude-sonnet-4-6

The Claude OCR engine sends each page image to the vision model:

# claude_engine.py
class ClaudeOCREngine:
    name = "claude"

    def ocr_page(self, png_bytes: bytes) -> str:
        settings = get_settings()
        client = anthropic.Anthropic(api_key=settings.anthropic_api_key)

        response = client.messages.create(
            model=settings.claude_vision_model,  # haiku-4-5
            max_tokens=4096,
            messages=[{
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": "image/png",
                            "data": base64.b64encode(png_bytes).decode(),
                        },
                    },
                    {
                        "type": "text",
                        "text": "Extract all text from this page as clean markdown. "
                                "Ignore navigation bars, footers, share buttons, "
                                "and other page chrome."
                    }
                ],
            }]
        )
        return response.content[0].text

The extraction call goes to Sonnet with a much larger max_tokens budget:

# llm.py (extraction)
async def _extract_async(self, doc: ParsedDocument) -> ExtractionResult:
    result = await self.provider.complete(
        [Message(role="system", content=SYSTEM),
         Message(role="user", content=user_prompt)],
        temperature=0.1,
        max_tokens=24000,  # extraction needs headroom; Sonnet handles it
    )

The LLMProvider abstraction routes through settings.claude_model (Sonnet) for text completions and settings.claude_vision_model (Haiku) for image passes. Two env vars, completely separate call paths.

The Bug That Made This Non-Obvious

I hit a debugging session that illustrates why you need to actually measure this instead of guessing.

My first extraction run produced this error:

json.decoder.JSONDecodeError: Unterminated string at char 373

Same position on every run, even after bumping max_tokens from 8192 to 16000. Suspicious. I captured the raw output:

$py -c "
result = provider.complete(...)
open('extract_raw.txt','w').write(result)
"

The saved file was clean, well-formed JSON. So _coerce_json was mangling it. Traced through:

# THE BUG
def _coerce_json(text: str) -> dict:
    if text.startswith("```"):
        text = text.split("```", 2)[1]  # <-- catastrophic

The extraction document contained code samples. Sonnet wrapped its JSON in a json ` fence—standard behavior—but the article body also had code blocks with ` fences inside the JSON string values. My naive split("`", 2) shredded the output at the first interior fence, producing an empty string.

Fix: strip the outer fence line-by-line instead of splitting:

def _coerce_json(text: str) -> dict:
    lines = text.strip().splitlines()
    # Strip leading ```json fence
    if lines and lines[0].startswith("```"):
        lines = lines[1:]
    # Strip trailing ``` fence
    if lines and lines[-1].strip() == "```":
        lines = lines[:-1]
    raw = "\n".join(lines)
    start = raw.find("{")
    end = raw.rfind("}")
    return json.loads(raw[start:end + 1])

Also switched the Claude provider to streaming to avoid token truncation on large extractions—the original non-streaming call was silently hitting the max_tokens ceiling without error.

What the Pipeline Produced

After the fix, one Haiku OCR pass + one Sonnet extraction call over the 26-page PDF produced 8 atomic concept drafts:

concurrency/concurrency-definition (diff 2) prereqs=[] Qs=4 concurrency/parallelism-definition (diff 2) prereqs=['concurrency/concurrency-definition'] Qs=4 concurrency/async-io-event-loop (diff 3) prereqs=['concurrency/concurrency-definition'] Qs=4 concurrency/race-conditions-shared-state (diff 3) prereqs=['concurrency/parallelism-definition'] concurrency/amdahls-law (diff 4) concurrency/ruby-gvl (diff 4) prereqs=[4 ancestors] concurrency/ruby-ractors (diff 5) concurrency/concurrency-decision-framework (diff 3)

Each with a definition, examples, interview questions, prerequisite links, and provenance tied back to the source URL. Total cost: $0.18.

The Generalizable Pattern

This isn't just about OCR. The same logic applies anywhere you're mixing vision and reasoning:

Classification/routing ("is this a blog post, academic paper, or tutorial?"): Haiku
Structured data extraction from forms/tables: Haiku or Sonnet depending on complexity
Summarization: Sonnet usually enough
Deep synthesis across multiple sources: Sonnet or Opus
High-stakes generation (interview prep, code review): Opus

The mistake is reaching for the strongest model by default and never questioning it. OCR especially—it's an unusually clear case where you're paying Opus prices for a task that's mostly pixel-to-character mapping.

If you're building on the Claude API and haven't audited which model you're using for which step, that's probably the first place to look. The gap between Haiku and Opus is 5× on input tokens. Over any real volume, that compounds fast.

Drafted by Claude Sonnet from my own Claude Code session transcript, then reviewed and edited before publishing.

DEV Community: yogeshchavan2008