Reduce OpenAI Costs by 80%: TOON Reality + Two-Stage Architecture

🎯 Introduction

TOON is popular but not always optimal for nested structures. Real savings come from architectural optimization.

This article includes:

• A brief TOON evaluation
• A two-stage architecture reducing costs by 80%
• A full fake-data pipeline example

🔍 TOON Summary

• Great for flat data
• Less efficient for nested structures
• JSON Compact often performs better
• Biggest savings come from sending less data to the LLM

---

⚡ Two-Stage Architecture

1️⃣ Minimal Extraction (LLM)

Only tender_id and bids extracted.

2️⃣ Local Enhancement (Zero Tokens)

Addresses, licenses, and numeric values processed locally.

---

📁 Fake Data Example Pipeline

1️⃣ input.md

AGENCY: Example Transportation Authority
PROJECT ID: 00-X00001
COUNTY: NORTHFIELD

**Bidder 1:** Alpha Infrastructure Group
Address: 101 Example Road, Eastville, EX 90001
Phone: [phone]
License: FAKE12345
Total Bid: $1,234,567.89

2️⃣ minimal_schema.json

{
  "type": "object",
  "properties": {
    "tender_id": {"type": "string"},
    "bids": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "firm": {"type": "string"},
          "amount": {"type": "string"}
        },
        "required": ["firm", "amount"]
      }
    }
  },
  "required": ["tender_id", "bids"]
}

3️⃣ llm_output.json

{
  "tender_id": "00-X00001",
  "bids": [
    {"firm": "Alpha Infrastructure Group", "amount": "$1,234,567.89"}
  ]
}

4️⃣ enhanced_output.json

{
  "tender_id": "00-X00001",
  "bids": [
    {
      "firm": "Alpha Infrastructure Group",
      "amount": {
        "raw": "$1,234,567.89",
        "numeric": 1234567.89,
        "currency": "USD"
      },
      "address": {
        "street": "101 Example Road",
        "city": "Eastville",
        "state": "EX",
        "zip": "90001"
      },
      "license": "FAKE12345",
      "phone": "[phone]"
    }
  ]
}

🧠 Conclusion

This architecture:

• Reduces cost by 80%
• Performs faster
• Scales effectively