DEV Community

yogeshchavan2008
yogeshchavan2008

Posted on

Cut Claude API Costs 80% by Splitting Vision and Reasoning Tasks

#ai #api #llm #claude

The Problem That Forced the Question

I was building an ingestion pipeline for a personal knowledge system—drop in a PDF, get structured OKF-format concept articles out the other side. Anthropic's vision models are genuinely good at reading image-only PDFs (scanned docs, Medium exports printed to PDF), so I wired them into my OCR layer.

Then I looked at the cost estimate for my sample: a 26-page image-only PDF. Running everything through claude-opus-4-8 would cost $1.00–$1.30 for a single document. Scale that to a few dozen docs and suddenly my personal knowledge base has a real operational cost.

The fix was obvious once I thought about it—but I hadn't thought about it.

The Insight: OCR Is Transcription, Not Reasoning

Vision OCR and concept extraction are fundamentally different tasks:

  • OCR: "What text is on this page?" Rote transcription. High-contrast text on a white background. No ambiguity. This does not need Opus.
  • Extraction: "Given this 15,000-character document about concurrency, identify the atomic concepts, their prerequisites, define their difficulty levels, and generate interview questions." This benefits from a stronger model.

Once I separated these mentally, the routing became obvious.

The Pricing Math

Current Claude API pricing (per 1M tokens):

Model Input Output
claude-opus-4-8 $5.00 $25.00
claude-sonnet-4-6 $3.00 $15.00
claude-haiku-4-5 $1.00 $5.00

Image tokens are roughly (width × height) / 750. My PDF rendered at 200 DPI produces Letter-size pages at ~1700×2200px—about 3.7MP—which hits near Opus's 4,800-token ceiling per page.

26 pages × ~4,800 image tokens per page = ~125,000 image tokens just for OCR.

Route Est. cost Notes
Opus for everything ~$1.10 wasteful
Haiku OCR + Opus extraction ~$0.20 OCR is cheap on Haiku
Haiku OCR + Sonnet extraction ~$0.18 sweet spot
Haiku for both ~$0.12 lower extraction quality

That's an 84% reduction by using Haiku for the vision calls and Sonnet for the one extraction call.

What This Looks Like in Code

In .env:

OCR_ENGINE=claude
CLAUDE_VISION_MODEL=claude-haiku-4-5
CLAUDE_MODEL=claude-sonnet-4-6
Enter fullscreen mode Exit fullscreen mode

The Claude OCR engine sends each page image to the vision model:

# claude_engine.py
class ClaudeOCREngine:
    name = "claude"

    def ocr_page(self, png_bytes: bytes) -> str:
        settings = get_settings()
        client = anthropic.Anthropic(api_key=settings.anthropic_api_key)

        response = client.messages.create(
            model=settings.claude_vision_model,  # haiku-4-5
            max_tokens=4096,
            messages=[{
                "role": "user",
                "content": [
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": "image/png",
                            "data": base64.b64encode(png_bytes).decode(),
                        },
                    },
                    {
                        "type": "text",
                        "text": "Extract all text from this page as clean markdown. "
                                "Ignore navigation bars, footers, share buttons, "
                                "and other page chrome."
                    }
                ],
            }]
        )
        return response.content[0].text
Enter fullscreen mode Exit fullscreen mode

The extraction call goes to Sonnet with a much larger max_tokens budget:

# llm.py (extraction)
async def _extract_async(self, doc: ParsedDocument) -> ExtractionResult:
    result = await self.provider.complete(
        [Message(role="system", content=SYSTEM),
         Message(role="user", content=user_prompt)],
        temperature=0.1,
        max_tokens=24000,  # extraction needs headroom; Sonnet handles it
    )
Enter fullscreen mode Exit fullscreen mode

The LLMProvider abstraction routes through settings.claude_model (Sonnet) for text completions and settings.claude_vision_model (Haiku) for image passes. Two env vars, completely separate call paths.

The Bug That Made This Non-Obvious

I hit a debugging session that illustrates why you need to actually measure this instead of guessing.

My first extraction run produced this error:

json.decoder.JSONDecodeError: Unterminated string at char 373
Enter fullscreen mode Exit fullscreen mode

Same position on every run, even after bumping max_tokens from 8192 to 16000. Suspicious. I captured the raw output:

$py -c "
result = provider.complete(...)
open('extract_raw.txt','w').write(result)
"
Enter fullscreen mode Exit fullscreen mode

The saved file was clean, well-formed JSON. So _coerce_json was mangling it. Traced through:

# THE BUG
def _coerce_json(text: str) -> dict:
    if text.startswith("```"):
        text = text.split("```", 2)[1]  # <-- catastrophic
Enter fullscreen mode Exit fullscreen mode

The extraction document contained code samples. Sonnet wrapped its JSON in a json ` fence—standard behavior—but the article body also had code blocks with ` fences inside the JSON string values. My naive split("`", 2) shredded the output at the first interior fence, producing an empty string.

Fix: strip the outer fence line-by-line instead of splitting:

def _coerce_json(text: str) -> dict:
    lines = text.strip().splitlines()
    # Strip leading ```json fence
    if lines and lines[0].startswith("```"):
        lines = lines[1:]
    # Strip trailing ``` fence
    if lines and lines[-1].strip() == "```":
        lines = lines[:-1]
    raw = "\n".join(lines)
    start = raw.find("{")
    end = raw.rfind("}")
    return json.loads(raw[start:end + 1])
Enter fullscreen mode Exit fullscreen mode

Also switched the Claude provider to streaming to avoid token truncation on large extractions—the original non-streaming call was silently hitting the max_tokens ceiling without error.

What the Pipeline Produced

After the fix, one Haiku OCR pass + one Sonnet extraction call over the 26-page PDF produced 8 atomic concept drafts:

concurrency/concurrency-definition (diff 2) prereqs=[] Qs=4
concurrency/parallelism-definition (diff 2) prereqs=['concurrency/concurrency-definition'] Qs=4
concurrency/async-io-event-loop (diff 3) prereqs=['concurrency/concurrency-definition'] Qs=4
concurrency/race-conditions-shared-state (diff 3) prereqs=['concurrency/parallelism-definition']
concurrency/amdahls-law (diff 4)
concurrency/ruby-gvl (diff 4) prereqs=[4 ancestors]
concurrency/ruby-ractors (diff 5)
concurrency/concurrency-decision-framework (diff 3)

Each with a definition, examples, interview questions, prerequisite links, and provenance tied back to the source URL. Total cost: $0.18.

The Generalizable Pattern

This isn't just about OCR. The same logic applies anywhere you're mixing vision and reasoning:

  • Classification/routing ("is this a blog post, academic paper, or tutorial?"): Haiku
  • Structured data extraction from forms/tables: Haiku or Sonnet depending on complexity
  • Summarization: Sonnet usually enough
  • Deep synthesis across multiple sources: Sonnet or Opus
  • High-stakes generation (interview prep, code review): Opus

The mistake is reaching for the strongest model by default and never questioning it. OCR especially—it's an unusually clear case where you're paying Opus prices for a task that's mostly pixel-to-character mapping.

If you're building on the Claude API and haven't audited which model you're using for which step, that's probably the first place to look. The gap between Haiku and Opus is 5× on input tokens. Over any real volume, that compounds fast.

Drafted by Claude Sonnet from my own Claude Code session transcript, then reviewed and edited before publishing.

Top comments (0)