DEV Community: Adamo Software

REST vs GraphQL: Has your opinion changed?

Adamo Software — Thu, 23 Jul 2026 09:47:39 +0000

Why GraphQL didn't replace REST - and why both still matter in modern API development.

A few years ago, many developers believed GraphQL would eventually replace REST. The promise was attractive: one flexible API, no over-fetching, no under-fetching, and better frontend developer experience.

But after years of production experience, the conversation has changed. Many teams still use REST successfully, while others have adopted GraphQL selectively rather than replacing everything.

So the question today is not "Which one is better?"

The better question is: Has your opinion about REST and GraphQL changed after building real systems?

Why has GraphQL become so popular?

When GraphQL was introduced, many developers saw it as a potential replacement for REST. The main reason was simple: modern applications were becoming more complex, but traditional APIs were not always flexible enough to support rapidly changing frontend requirements.

With REST, clients often face two common problems:

Over-fetching: The API returns more data than the client needs.
Under-fetching: The client needs to call multiple endpoints to collect related data.

For example, a dashboard might need user information, orders, recommendations, and analytics. A REST approach may require several API requests, while GraphQL allows clients to request all required data through a single query.

Beyond flexible data fetching, GraphQL also improved developer experience through:

A strongly typed schema
Better API documentation
Easier frontend-backend collaboration
More control over response structures

For applications with complex interfaces and rapidly changing requirements, these benefits made GraphQL extremely attractive.

Where GraphQL actually shines

GraphQL works best in situations where flexibility matters more than simplicity.

1. Complex applications with many data relationships

Applications such as marketplaces, social platforms, and travel platforms often need data from multiple sources.

For example, a travel booking interface may combine:

Flight details
Hotel information
Customer profiles
Reviews
Payment status

GraphQL can provide this data through a single request instead of requiring multiple API calls.

2. Multiple clients with different needs

Modern products often support:

Web applications
Mobile apps
Partner platforms

Each client may require different data. Instead of creating separate REST endpoints for every use case, GraphQL allows each client to request exactly what it needs.

3. Fast product iteration

When product requirements change frequently, GraphQL can reduce dependency between frontend and backend teams. Developers can often request existing data fields without waiting for new API endpoints to be created.

However, these advantages come with trade-offs.

Why is REST still everywhere?

Despite GraphQL's popularity, REST remains the default choice for many APIs because it is simple, predictable, and well understood.

1. REST works naturally with HTTP

REST follows standard web principles:

GET /products
POST /orders
DELETE /users/123

Developers already understand:

HTTP methods
Status codes
Headers
Authentication patterns

This makes REST easier to build, maintain, and consume.

2. REST has better caching support

One major advantage of REST is its compatibility with existing HTTP caching mechanisms.

A request like:

GET /products/123

can easily be cached using:

Browser caching
CDN caching
Cache-Control headers
ETags

GraphQL can also use caching, but it usually requires additional strategies and tooling.

3. Many applications don't need GraphQL

GraphQL solves specific problems, but not every application has those problems.

For:

Internal tools
Simple SaaS applications
CRUD-based systems

A well-designed REST API may be faster to build and easier to maintain.

Sometimes simplicity is the better engineering decision.

Where GraphQL gets complicated

GraphQL provides flexibility, but that flexibility introduces new challenges.

1. More complex caching

Because GraphQL commonly uses a single endpoint: POST /graphql

different queries can request completely different data.

This makes caching more difficult compared to traditional REST endpoints.

2. Query performance risks

A poorly designed GraphQL query can request deeply nested data and create expensive database operations.

For example:

User
→ Orders
→ Products
→ Reviews

To prevent performance issues, teams need additional controls such as:

Query limits
Rate limiting
Query complexity analysis

3. Higher backend complexity

GraphQL requires teams to carefully manage:

Schema design
Resolvers
Authorization
Data loading performance

A poorly implemented GraphQL API can become harder to maintain than a REST API.

REST vs GraphQL: The decision framework

The choice should depend on your application's requirements, not popularity.

Choose REST when:

Your API is resource-based
You need simple and predictable endpoints
You want strong HTTP caching
External developers will consume your API
Your application has straightforward data requirements

Choose GraphQL when:

Your application has complex data relationships
Multiple clients require different data
Frontend flexibility is a priority
Your team can handle additional complexity

Conclusion

The REST vs GraphQL debate has changed. The best choice depends on your product requirements, team expertise, and long-term goals. Instead of asking "REST or GraphQL?", a better question is:

Which API design approach helps us build a system that is easier to maintain, scale, and evolve?

How to Design Secure FHIR APIs for EHR/EMR Integration

Adamo Software — Fri, 17 Jul 2026 08:34:14 +0000

FHIR (Fast Healthcare Interoperability Resources) has become the standard for integrating EHR and EMR systems, enabling healthcare providers to exchange patient data more efficiently than ever. But every API that exposes electronic health records also expands the attack surface.

Even a single vulnerable API endpoint can lead to unauthorized access or the exposure of sensitive patient information. Recent incidents, such as the data breach involving the Singapore Land Authority's vendor that exposed the personal information of around 70,000 individuals, serve as a reminder that even trusted third-party systems can become entry points for sensitive data exposure.

In this article, we'll explore the key security principles and best practices for designing FHIR APIs that enable seamless EHR/EMR integration while safeguarding sensitive healthcare data.

01. Why FHIR API Security Matters
FHIR (Fast Healthcare Interoperability Resources) has become the leading standard for exchanging healthcare data between EHR/EMR systems and modern healthcare applications. From patient portals and telemedicine platforms to laboratory systems and mobile health apps, FHIR APIs enable seamless interoperability across the healthcare ecosystem.

However, increased connectivity also comes with increased risk. Every API that exposes electronic health records becomes a potential entry point for cyberattacks if it isn't properly secured.

In healthcare, the stakes are even higher. Beyond financial losses, a compromised FHIR API can expose protected health information (PHI), disrupt clinical operations, and damage patient trust. That's why API security should be considered a core part of healthcare system architecture rather than an afterthought.

02. Common Security Challenges in EHR/EMR Integration
Designing secure FHIR APIs starts with understanding the most common security risks that arise during EHR/EMR integration.

Unauthorized access: Not every authenticated user should have access to every patient record. Without proper authorization controls, healthcare professionals may unintentionally gain access to data outside their responsibilities.
Excessive data exposure: Sharing more data than necessary increases privacy risks and the potential impact of a breach.
Third-party integrations: Healthcare platforms often exchange data with laboratories, pharmacies, insurance providers, wearable devices, and telemedicine applications.
Lack of monitoring: Without comprehensive audit logs, organizations may struggle to detect suspicious activities, investigate security incidents, or demonstrate compliance during security audits.

03. How to design secure FHIR APIs
Building secure FHIR APIs requires security to be integrated into every stage of the API lifecycle.

Implement strong authentication and authorization

Adopt industry standards such as OAuth 2.0 and OpenID Connect to verify user identities. Beyond authentication, implement fine-grained authorization based on user roles, assigned patients, departments, or patient consent. Applying the principle of least privilege ensures users only access the information necessary for their responsibilities.

Protect sensitive healthcare data

Encrypt data both in transit using TLS and at rest using strong encryption standards. APIs should also follow the principle of data minimization by returning only the information required for a specific workflow instead of exposing entire patient records.

Secure every API request

Validate all incoming requests to prevent injection attacks and malformed payloads. API gateways can further enhance security by enforcing authentication, rate limiting, request validation, and centralized security policies before requests reach backend systems.

Monitor and audit API activities

Every interaction with patient data should be logged, including who accessed the data, when the access occurred, and what actions were performed. Continuous monitoring and audit logging help organizations detect abnormal behavior, investigate incidents, and support regulatory compliance.

04. Common mistakes to avoid
Even well-designed healthcare systems can introduce security risks through common implementation mistakes.

Returning full FHIR resources when only a few fields are required.
Relying solely on user roles without implementing fine-grained access controls.
Logging sensitive patient information in application or server logs.
Using long-lived access tokens without proper expiration or rotation policies.
Skipping input validation for API requests.
Missing rate limiting and API throttling, leaving endpoints vulnerable to abuse.
Treating security as a compliance checklist rather than an integral part of API design.

Avoiding these issues early in development significantly reduces security risks and creates a stronger foundation for scalable healthcare integrations.

05. Adamo Software - Your Vietnam-based trusted partner for healthcare software development

Developing secure healthcare applications requires more than technical expertise—it demands a deep understanding of interoperability standards, regulatory requirements, and security best practices.

At Adamo Software, we help healthcare organizations design and develop secure, scalable, and compliant software solutions tailored to their business needs. Our engineering teams have extensive experience building healthcare platforms, integrating EHR/EMR systems, implementing FHIR-based APIs, and developing telemedicine, patient engagement, and healthcare management solutions.

Also, we hold ISO 27001 and ISO 9001:2015 certifications. Every engagement includes PDPA-aligned data handling, NDAs signed, and full IP and source-code ownership transferred to the client from day one.

Partner with us for secure healthcare software development.

How we optimized our LLM pipeline to cut token usage by 70%

Adamo Software — Mon, 06 Jul 2026 04:05:24 +0000

Most teams assume the fastest way to reduce AI costs is to switch to a smaller model. In reality, that's often the last thing you should do. Within a few weeks we noticed three problems:

API costs were increasing every day.
Response latency became inconsistent.
Most requests didn't actually need the most capable (and most expensive) model.

Downgrading to a cheaper model reduced costs, but it also reduced answer quality. Instead, we optimized the entire pipeline. The result was a significant reduction in token usage and API spend while keeping response quality almost unchanged. Here's what made the biggest difference.

Stop sending everything to the biggest model

One of the biggest cost drivers wasn't the model itself - it was how we used it.

Tasks like summarizing text, extracting keywords, formatting JSON, or rewriting an email don't require advanced reasoning. Running these requests on your most expensive model is like using a supercomputer to open a spreadsheet. Instead, introduce a lightweight routing layer before calling the LLM.

                User Request 

                      │ 

                      ▼ 

          ┌────────────────────┐ 

          │ Complexity Router   │ 

          └────────────────────┘ 

              │            │ 

      Simple Task     Complex Task 

              │            │ 

              ▼            ▼ 

      GPT-4.1 Mini      GPT-5

A simple classifier—or even a few rule-based checks—can decide which model is appropriate.

This optimization alone can dramatically reduce API costs without affecting the user experience.

Shorter prompts are usually better prompts

Long prompts are expensive.

Unfortunately, prompts tend to grow over time. Every new feature adds another instruction, another example, or another formatting rule. Eventually, the prompt becomes larger than the user's actual question.

The shorter version is easier to maintain, consumes fewer tokens, and often produces the same quality of responses. Instead of continuously adding instructions, periodically review your prompts and remove anything that no longer provides measurable value.

Rule of thumb: Every token in your prompt is a token you pay for.

Add Semantic Caching

Traditional caching only works when two requests are identical.LLM applications rarely receive identical requests.

For example: "How do I request my password?" vs. "I forgot my password".

The wording is different, but the intent is the same. This is where semantic caching becomes useful.

       User Question
             │
             ▼
     Generate Embedding
             │
             ▼
      Vector Similarity
             │
    ┌────────┴─────────┐
    │                  │
 Cache Hit         Cache Miss
    │                  │
    ▼                  ▼

Cached Response Call LLM

Instead of comparing text character by character, compare the meaning of each request.

If a sufficiently similar response already exists, return it immediately instead of calling the LLM again.

For customer support, internal knowledge bases, or FAQ assistants, semantic caching can eliminate a large percentage of duplicate requests.

Send less context to the model

A common misconception is that more context always leads to better answers. In reality, too much context often has the opposite effect.

Many RAG pipelines simply retrieve the top 10 or 20 document chunks and send everything to the model.

Knowledge Base
│
Retrieve Top 10 Chunks
│
▼
LLM

This increases token usage, latency, and cost. Instead, retrieve broadly, then filter aggressively.

Knowledge Base
│
Hybrid Search
│
Reranker
│
Top 3 Chunks
│
▼
LLM

The model receives fewer tokens, but those tokens are significantly more relevant. In many cases, this not only reduces costs but also improves answer quality by removing distracting information.

Ask for structured output

Structured outputs offer several advantages:

fewer output tokens
faster responses
easier parsing
more reliable downstream automation

Sometimes we accidentally ask the model to generate much more text than we actually need.

Imagine building a travel booking platform.

If your application only needs destination, travel dates, and budget, asking the model to "explain its reasoning" wastes both time and tokens. Instead of this: Explain your reasoning before generating the final answer. Ask for structured data directly.

{
"destination": "Tokyo",
"duration": "5 days",
"budget": "$1500"
}

Final thoughts

Optimizing an LLM application isn't just about choosing the right model—it's about designing a smarter pipeline around it.

In our experience, the biggest cost savings didn't come from switching to a smaller model. They came from reducing unnecessary tokens, routing requests more intelligently, caching repeated queries, and providing only the context the model actually needed.

As LLM applications move from prototypes to production, these optimizations become increasingly important. A few architectural improvements can significantly reduce costs, improve latency, and create a better user experience—all without compromising response quality.

Start by measuring where your tokens are going. You'll likely find that the biggest opportunities aren't inside the model, but in everything around it.

05 common pitfalls when working with GDS in travel technology

Adamo Software — Mon, 15 Jun 2026 02:11:02 +0000

GDS integrations sit at the core of many travel platforms, yet they remain one of the most technically challenging components to implement and maintain. Most GDS environments are built on decades-old infrastructure, where modern JSON-based services often coexist with legacy protocols, XML schemas, EDIFACT messages, and proprietary business logic. Developers must navigate challenges related to session management, asynchronous workflows, fare construction, inventory synchronization, rate limiting, and inconsistent data models across suppliers.

In this article, Adamo Software will break down five common GDS integration pitfalls and share practical tips to help engineering teams avoid them.

Treating GDS APIs like standard REST services

Most GDS platforms run on legacy, stateful architectures rather than modern stateless APIs. A single booking transaction typically involves multiple steps—availability search, fare validation, PNR creation, and ticket issuance—often managed through persistent sessions and complex business rules.

Practical tip: Treat GDS integrations as workflow orchestration problems instead of simple API calls. Build a dedicated integration layer to manage sessions, retries, idempotency, and error handling independently from the core application.

Failing to normalize data across multiple suppliers

Different suppliers represent the same information in different ways. Baggage policies, fare rules, seat attributes, and ancillary services may vary across GDS providers, NDC APIs, and direct airline connections, creating inconsistent user experiences and increasing development complexity.

Practical tip: Implement a canonical data model and a dedicated normalization layer that transforms supplier-specific schemas into standardized internal entities. This approach simplifies downstream development and reduces the impact of provider changes.

Underestimating API limits and scalability requirements

GDS providers typically enforce strict rate limits, transaction quotas, and usage-based pricing. High search volumes, inefficient retries, or duplicate requests can quickly drive up costs and affect system performance during peak travel periods.

Practical tip: Optimize API consumption with caching, request deduplication, and intelligent search strategies. Monitor API usage continuously and implement rate limiting, circuit breakers, and autoscaling to ensure reliability at scale.

The NDC gap

Traditional GDS channels do not always provide access to the full range of airline content. Dynamic pricing, ancillary services, branded fares, and personalized offers are increasingly distributed through NDC APIs and direct airline channels.

Practical tip: Design your architecture to support multiple content sources from the start. An abstraction layer that combines GDS, NDC, and direct APIs can help ensure broader inventory coverage and future-proof your platform.

Ignoring real-time inventory and pricing volatility

Travel inventory and pricing change constantly across multiple distribution channels. The fare or seat availability displayed during search may no longer be valid at checkout, leading to pricing discrepancies and booking failures.

Practical tip: Validate availability and reprice offers immediately before booking confirmation. Minimize the time between search and purchase, and design the user experience to handle price changes transparently.

Building GDS integrations starts with the right architecture

Successful GDS integrations require more than connecting to an API. From legacy workflows and fragmented data models to NDC adoption and real-time pricing volatility, each challenge has a direct impact on system performance, scalability, and user experience. By designing for flexibility, observability, and multi-source distribution from the outset, travel technology teams can reduce integration risks and build platforms that are ready for the future of travel distribution.

Strict-schema LLM outputs: what we learned shipping to a HIPAA environment

Adamo Software — Mon, 11 May 2026 09:47:37 +0000

Most LLM tutorials show structured output as a one-liner: pass a Pydantic model, get back validated JSON, ship it. In production with PHI on the line, that one-liner is the easy 20% of the problem. The other 80% is what happens when the schema validates but the data is still wrong, when the model returns a JSON that passes type checks but contains hallucinated content, or when a field that should be redacted slips through because the schema only knew about its shape.

I worked on a clinical documentation pipeline last year where the LLM was extracting structured data from doctor-patient conversations. The schema validated. The integration tests passed. Then a real visit transcript came in where the patient mentioned a family member's medication and the LLM helpfully attributed it to the patient. Schema-valid. Clinically wrong. PHI-adjacent enough to trigger a compliance review.

This post covers the five patterns we ended up adopting. The HIPAA context made these patterns mandatory for us, but they apply to any production LLM pipeline where downstream systems trust the output.

Why "valid JSON" is not enough

Native structured outputs from OpenAI, Anthropic, and Google now guarantee schema conformance at the generation level. The API physically cannot return JSON that violates your schema. This is a real improvement over the prompt-and-pray approach from 2023.

The problem is that schema conformance is a syntactic guarantee, not a semantic one. A schema like this is fine:

from pydantic import BaseModel, Field
from typing import Optional

class PatientMedication(BaseModel):
    name: str = Field(..., min_length=1)
    dosage: str
    frequency: str
    prescribed_for_patient: bool

The model will return JSON that fits this shape every time. What it does not guarantee is that name is actually a medication, that dosage is parseable, or that prescribed_for_patient reflects what was actually said. In healthcare, "valid JSON, wrong field assignment" is a clinical incident, not a parsing bug.

The patterns below assume you already use native structured outputs and Pydantic (or Zod for JS). They go past schema validation into the harder problem: making sure the structured output is also trustworthy.

Pattern 1: Constrain enums brutally, never accept freeform strings

The biggest single source of downstream bugs we saw was string fields that should have been enums. The model would return "twice daily", "2x per day", "BID", "every 12 hours" for the same input. All valid strings. All meaning the same thing. All breaking the billing system that expected one of five canonical values.

The fix is to type every categorical field as an enum and let the schema enforce the closed set:

from enum import Enum

class Frequency(str, Enum):
    ONCE_DAILY = "once_daily"
    TWICE_DAILY = "twice_daily"
    THREE_TIMES_DAILY = "three_times_daily"
    FOUR_TIMES_DAILY = "four_times_daily"
    AS_NEEDED = "as_needed"
    OTHER = "other"

class PatientMedication(BaseModel):
    name: str = Field(..., min_length=1, max_length=200)
    dosage: str = Field(..., max_length=100)
    frequency: Frequency
    frequency_raw: Optional[str] = Field(
        None,
        description="Original text from the transcript if frequency is OTHER"
    )

Two things matter here. First, the enum is the closed set the downstream system knows how to handle. The model has to pick one. Second, there's an OTHER escape hatch with a frequency_raw field that captures the original text. This is the difference between "schema validation forced the model to lie" and "schema validation surfaced an edge case for human review."

We saw a measurable drop in downstream parsing errors after we converted every categorical field to this pattern. Roughly 8% of medication records had frequency strings the billing system couldn't parse before. After the enum migration, that dropped to under 0.5%, and the remaining cases all landed in OTHER with the raw text preserved.

Pattern 2: Confidence and source-attribution fields

The schema-valid-but-wrong problem comes mostly from the model being confidently wrong. The mitigation is to make the model state its own uncertainty inside the schema. This sounds soft. It is not. When the model has to output a confidence score and an attribution field for every claim, the failure modes change.

from typing import List, Literal

class Citation(BaseModel):
    text_span: str = Field(..., description="Exact quote from transcript")
    speaker: Literal["doctor", "patient", "other"]

class ExtractedFact(BaseModel):
    field_name: str
    value: str
    confidence: float = Field(..., ge=0.0, le=1.0)
    citations: List[Citation] = Field(..., min_length=1)
    requires_human_review: bool

class VisitExtraction(BaseModel):
    medications: List[ExtractedFact]
    diagnoses: List[ExtractedFact]
    follow_up_actions: List[ExtractedFact]

Three rules we ended up enforcing:

Confidence is a float, not a category. Categorical confidence ("high", "medium", "low") gets gamed by the model into all-high.
Citations are required. Every extracted fact must include the exact span from the source. If the model cannot cite, it should not extract. This is much harder to fake.
requires_human_review is computed by the model, not by you. Adding this field changes how the model reasons about its own outputs. We saw the model start flagging genuinely ambiguous cases that we had not anticipated.

The downstream system then routes anything with confidence < 0.8 or requires_human_review == True to a review queue. This is the same pattern Collin Wilkins describes in Structured outputs for real pipelines, and it generalizes well beyond healthcare.

Pattern 3: Post-validation passes that schemas cannot express

Some validations are impossible to express in JSON Schema. A medication dosage of "500mg" is a valid string. A dosage of "500000mg" is also a valid string, but it would kill someone. The schema cannot tell the difference.

We layered a post-validation pass on top of Pydantic that runs domain-specific checks:

from pydantic import BaseModel, field_validator
import re

class PatientMedication(BaseModel):
    name: str
    dosage_amount: float
    dosage_unit: Literal["mg", "g", "mcg", "ml", "units"]
    frequency: Frequency

    @field_validator("dosage_amount")
    @classmethod
    def reasonable_dosage(cls, v: float, info) -> float:
        if v <= 0 or v > 10000:
            raise ValueError(
                f"Dosage {v} outside reasonable range. "
                "Flag for review."
            )
        return v

The validator does not just reject bad values, it raises a typed error that the pipeline catches and routes to the review queue with context. The model is not punished for surfacing the edge case; it is rewarded because the review queue captures the original output and the validator error together.

A second class of validations needs cross-field logic. For example, if route == "topical" then dosage_unit cannot be "units". These go in model_validator decorators rather than field_validator.

Pattern 4: PHI redaction as a separate, schema-enforced step

This is the pattern specific to HIPAA, but the principle generalizes to anything with sensitive data. The instinct is to ask the LLM to extract structured data and redact PHI in the same call. This is a mistake. The model is good at one or the other, not both at once.

We split into two passes with separate schemas:

# Pass 1: Extract structured clinical data, PHI preserved
class RawExtraction(BaseModel):
    medications: List[PatientMedication]
    diagnoses: List[Diagnosis]
    raw_transcript: str

# Pass 2: De-identify, output PHI-free version
class PHISpan(BaseModel):
    category: Literal[
        "name", "date", "location", "phone", 
        "address", "mrn", "other"
    ]
    text: str
    start: int
    end: int

class DeidentifiedOutput(BaseModel):
    redacted_transcript: str
    phi_spans_found: List[PHISpan]
    safe_to_log: bool

The first pass runs in a HIPAA-controlled environment with a BAA in place. The second pass is the gate before anything leaves that environment, including before any logs or traces reach observability tools. This separation matters because research on LLM-based PHI annotation shows that combining extraction and de-identification in one prompt degrades both tasks significantly.

For organizations building this kind of pipeline, the AI medical scribe architecture guide covers the broader integration patterns including FHIR write-back and EHR boundaries. The point for this post is narrower: never let a single model call be responsible for both producing structured output and protecting sensitive data. The schemas have to be separate, and the second pass has to be the gate.

Pattern 5: Replay and diff outputs against schema versions

Schemas evolve. A field that was optional becomes required, an enum gains a new value, a confidence threshold tightens. In a regulated environment, you cannot just deploy a new schema version and lose the ability to reproduce older outputs.

We version schemas the way we version database migrations:

class SchemaVersion(BaseModel):
    version: str  # semver: "1.2.0"
    deployed_at: datetime
    breaking_changes: List[str]

class StoredExtraction(BaseModel):
    schema_version: str
    raw_llm_output: str  # The JSON string before parsing
    parsed: Dict
    extraction_timestamp: datetime
    model_id: str  # "gpt-4o-2024-08-06"

    def replay_with_current_schema(
        self, current_schema_class
    ) -> tuple[Optional[BaseModel], List[str]]:
        """Try to parse stored output with current schema, return errors."""
        try:
            parsed = current_schema_class.model_validate_json(
                self.raw_llm_output
            )
            return parsed, []
        except ValidationError as e:
            return None, [str(err) for err in e.errors()]

This sounds like over-engineering. It is until the day the compliance team asks you to produce all extractions that used a specific schema version because a downstream system had a bug. With this pattern, you store the raw output and can re-parse it against any historical schema. Without it, you have to re-run the LLM, which is expensive, non-deterministic, and arguably a compliance violation on its own (because you are now producing new PHI processing events for an audit).

Schema versioning also makes A/B testing schemas safe. You can deploy v1.2.0 to 10% of traffic, compare extraction quality against v1.1.0, and roll back without losing data. There's a good Dev.to thread on Pydantic patterns for production that covers some of this, though the regulated-environment angle is less common.

What this looks like in production

Putting the five patterns together, our extraction pipeline became:

LLM call with native structured outputs and a versioned Pydantic schema (Pattern 1, Pattern 5)
Pydantic validation catches schema violations (always passes thanks to native structured outputs)
Post-validation pass with domain-specific range checks and cross-field logic (Pattern 3)
Confidence and review-flag check routes uncertain extractions to human queue (Pattern 2)
Separate de-identification pass with PHI schema before anything leaves the controlled environment (Pattern 4)
Storage with raw output and schema version for replay (Pattern 5)

The cost is roughly 2x the latency and 1.5x the API spend per visit compared to a naive single-call extraction. The benefit is that the compliance team stopped asking "how do you know the output is correct" and started asking smaller, specific questions we could answer with audit trails.

If you are building anything similar, the order I would recommend: enums first (cheapest fix with the biggest quality jump), then confidence and citations (forces the model to reason about its own uncertainty), then post-validation, then schema versioning, then the de-identification split if you have a regulated environment. Doing them in this order means you ship value at every step rather than waiting until everything is in place.

Wrapping up

Strict schemas are necessary but not sufficient for production LLM outputs. The schema makes the JSON valid; the patterns above make the content trustworthy. In a HIPAA environment that distinction is the difference between shipping and getting blocked at compliance review. In other environments it is the difference between an LLM feature that ships once and one that survives a year of production traffic.

The patterns are not specific to healthcare. Replace "PHI" with "customer financial data" or "personal information" and the same five patterns apply. The constraint of working with PHI just made it cheaper for us to do this work because the alternative was much more expensive.

Tool-use API design for LLMs: 5 patterns that prevent agent loops and silent failures

Adamo Software — Tue, 05 May 2026 02:53:33 +0000

In July 2025, a developer's Claude Code instance hit a recursion loop and burned through 1.67 billion tokens in 5 hours, generating an estimated $16,000 to $50,000 in API charges before anyone noticed. The agent did not crash. It did not throw an error. It just kept calling tools, getting confused, calling more tools, and silently accumulating cost. Old software crashes. LLM agents spend.

This is the failure mode most teams discover the hard way. You design a clean tool interface, the agent works in your test environment, you ship it to production, and three weeks later something hits an edge case that sends the agent into a loop. The patterns below are what we have used to prevent agent loops and silent failures across production LLM systems handling thousands of tool calls per day. None of them are about better prompts. They are about better tool design.

Why agent loops happen in the first place

Before the patterns, it helps to understand the failure mode precisely.

An LLM agent receives a user request, reasons about which tool to call, calls it, gets a result, reasons about whether the goal is achieved, and either responds or calls another tool. This loop is supposed to terminate when the goal is achieved or the model decides no further action is needed.

It does not terminate when:

The tool result is ambiguous. The model cannot tell whether the call succeeded, so it tries again with slightly different parameters.
The tool fails silently. The model receives a non-error response that does not actually contain the data it needed, so it interprets this as "I should retry."
The tool returns conflicting information. Two consecutive calls return different results, the model loses confidence in either, and tries to "verify" by calling more tools.
The model misreads its own previous output. With long context windows, the model sees a previous tool call's result, forgets it already processed that result, and re-processes it as new information.

Every one of these is preventable with tool design. The model is not the problem. The interface is.

Pattern 1: Make every tool result self-describing

The single most common cause of agent loops is tool results that the model cannot interpret without making assumptions.

A bad tool result:

{
  "results": [
    {"id": "h_1234", "name": "Hotel Granbell", "price": 128},
    {"id": "h_5678", "name": "Shibuya Stream", "price": 142}
  ]
}

The model now has to assume what this means. Are these the only matches? Are there more? Is the search complete? What was searched for? When the model gets confused (and at scale, it always gets confused eventually), it calls the tool again "to verify."

A self-describing tool result:

{
  "status": "success",
  "search_id": "srch_abc123",
  "query_summary": {
    "destination": "Shibuya, Tokyo",
    "check_in": "2026-07-12",
    "check_out": "2026-07-15",
    "guests": 1,
    "max_price": 150
  },
  "results": [
    {"id": "h_1234", "name": "Hotel Granbell", "price": 128, "currency": "USD"},
    {"id": "h_5678", "name": "Shibuya Stream", "price": 142, "currency": "USD"}
  ],
  "total_matches": 2,
  "is_complete": true,
  "next_action_hint": "User has 2 valid options. Present both with prices. Do not search again unless user changes parameters."
}

The is_complete: true and next_action_hint fields are the critical additions. The model can now read this result, understand that the search is finished, and know what to do next without re-querying. The query_summary echo lets the model verify it called the tool with the right parameters.

The next_action_hint is unconventional but extremely effective. It is a short instruction included in the tool response that tells the model what state the conversation is in. Think of it as the tool nudging the model toward correct loop termination.

# Wrapping tools to inject next_action_hint
def with_action_hint(tool_func):
    def wrapper(*args, **kwargs):
        result = tool_func(*args, **kwargs)
        result['next_action_hint'] = derive_hint(result)
        return result
    return wrapper

def derive_hint(result):
    if result['status'] == 'success' and result['total_matches'] == 0:
        return "No matches. Inform user and ask for relaxed criteria. Do not retry."
    if result['status'] == 'success' and result['total_matches'] > 0:
        return f"Found {result['total_matches']} matches. Present to user. Do not search again unless parameters change."
    if result['status'] == 'error':
        return f"Tool failed: {result['error']}. Inform user. Do not retry without user input."
    return "Process result and decide next step."

Implementing this across our tool surface reduced retry-driven loops by approximately 60% in production.

Pattern 2: Distinguish between "no results" and "tool failure"

The second most common cause of agent loops: ambiguous failure states.

A search that returns zero matches is a successful tool call. A search that timed out is a failed tool call. To the LLM, both can look identical if the tool just returns an empty results array.

# Bad: indistinguishable from no results
def search_hotels(query):
    try:
        results = supplier_api.search(query)
        return {"results": results}
    except Exception:
        return {"results": []}  # silent failure

# Good: explicit status with retry guidance
def search_hotels(query):
    try:
        results = supplier_api.search(query, timeout=5)
        return {
            "status": "success",
            "results": results,
            "total_matches": len(results),
            "retryable": False,
        }
    except SupplierTimeout:
        return {
            "status": "error",
            "error_type": "timeout",
            "error_message": "Supplier API did not respond within 5 seconds.",
            "retryable": True,
            "retry_after_ms": 2000,
            "max_retries_remaining": get_retry_budget(query),
        }
    except SupplierAuthError:
        return {
            "status": "error",
            "error_type": "auth",
            "error_message": "API authentication failed.",
            "retryable": False,
            "user_facing_message": "We're having trouble accessing hotel data. Please try again later.",
        }
    except RateLimitError as e:
        return {
            "status": "error",
            "error_type": "rate_limit",
            "error_message": f"Rate limit hit. Reset in {e.reset_seconds}s.",
            "retryable": True,
            "retry_after_ms": e.reset_seconds * 1000,
        }

The retryable flag is doing real work. When it is false, the LLM knows there is no point retrying and will inform the user instead. When it is true, the LLM has a structured retry path with explicit limits.

Without this pattern, an authentication failure that looks like an empty result set causes the model to try increasingly creative parameter combinations to "find results," consuming tokens and producing nothing.

Pattern 3: Enforce a hard call budget at the orchestrator level

No matter how well-designed your tools are, the model will occasionally enter a loop. The orchestrator must enforce a hard ceiling.

class AgentOrchestrator:
    def __init__(self, max_tool_calls=15, max_total_cost_usd=0.50):
        self.max_tool_calls = max_tool_calls
        self.max_total_cost_usd = max_total_cost_usd
        self.calls_made = 0
        self.total_cost_usd = 0

    async def run_agent_turn(self, user_message, conversation_history):
        history = conversation_history + [{"role": "user", "content": user_message}]

        while self.calls_made < self.max_tool_calls:
            if self.total_cost_usd >= self.max_total_cost_usd:
                return self._cost_limit_response()

            response = await call_llm(history, tools=self.tools)
            self.total_cost_usd += response.cost_usd

            if not response.tool_calls:
                # Model produced final response, exit loop
                return response.content

            for tool_call in response.tool_calls:
                self.calls_made += 1
                tool_result = await self.execute_tool(tool_call)
                history.append({"role": "tool", "content": tool_result})

        # Hit call limit. Force a final response.
        return await self._force_final_response(history)

    async def _force_final_response(self, history):
        # Add explicit instruction and call LLM with tools=None
        history.append({
            "role": "system",
            "content": "Tool call limit reached. Produce a final response to the user "
                       "based on information already gathered. Do not request more tools."
        })
        response = await call_llm(history, tools=None)
        return response.content

Two safeguards here. First, max_tool_calls prevents infinite loops by capping iterations. Fifteen is our default for booking workflows. Anything more than that is almost always a sign the agent is confused, not productive. Second, max_total_cost_usd is a financial circuit breaker. Even if the agent finds creative ways to make many tool calls, it cannot spend more than the per-conversation budget.

When the limit is hit, the orchestrator does not just return an error. It calls the LLM one more time with tools=None, forcing it to produce a final response from whatever it has gathered. This is much better UX than "Sorry, agent failed."

For high-volume systems, also implement per-tenant rate limiting. The single-developer Claude Code incident burned $16-50K because there was no per-account ceiling. Production systems need both per-conversation and per-tenant limits.

Pattern 4: Detect repeated calls and short-circuit them

Even with call budgets, agents waste budget by repeating the same call with minor variations. The fix is a deduplication layer at the orchestrator.

import hashlib
import json

class ToolCallDeduplicator:
    def __init__(self, window_size=5):
        self.recent_calls = []
        self.window_size = window_size

    def is_duplicate(self, tool_name, arguments):
        signature = self._signature(tool_name, arguments)
        is_dup = any(call == signature for call in self.recent_calls)
        self.recent_calls.append(signature)
        if len(self.recent_calls) > self.window_size:
            self.recent_calls.pop(0)
        return is_dup

    def _signature(self, tool_name, arguments):
        # Normalize arguments for comparison
        normalized = json.dumps(arguments, sort_keys=True, default=str)
        return f"{tool_name}:{hashlib.sha256(normalized.encode()).hexdigest()[:16]}"


# In the orchestrator
async def execute_tool(self, tool_call):
    if self.deduplicator.is_duplicate(tool_call.name, tool_call.arguments):
        return {
            "status": "duplicate_call_blocked",
            "message": (
                f"This exact {tool_call.name} call was made earlier in this conversation "
                f"with the same arguments. The previous result is already in your context. "
                f"Use it instead of calling again."
            ),
            "retryable": False,
        }

    return await self._actually_execute(tool_call)

When the model calls the same tool with the same arguments twice in a 5-call window, the orchestrator returns a structured "this is a duplicate" message instead of executing again. The model sees this and almost always recovers, often by referring back to the earlier result.

This pattern caught about 8% of calls in our production systems. Eight percent of total tool calls were unnecessary repeats. Blocking them saved both cost and latency.

A subtle detail: the deduplication signature should be lossy enough to catch near-duplicates. We use exact argument matching, but for some tools (search queries that differ only in word order), a normalization step before hashing would catch more.

Pattern 5: Parameter validation at the boundary, not inside the LLM

The slowest path to detecting a bad tool call is letting the LLM make it, the tool execute it, and the failure propagate back. The fastest path is validating parameters before the tool runs.

from pydantic import BaseModel, Field, validator
from datetime import date, timedelta


class SearchHotelsArgs(BaseModel):
    destination: str = Field(min_length=2, max_length=100)
    check_in: date
    check_out: date
    guests: int = Field(ge=1, le=20)
    max_price: float = Field(gt=0, le=10000)

    @validator('check_in')
    def check_in_not_in_past(cls, v):
        if v < date.today():
            raise ValueError(f"check_in date {v} is in the past")
        return v

    @validator('check_out')
    def check_out_after_check_in(cls, v, values):
        if 'check_in' in values and v <= values['check_in']:
            raise ValueError("check_out must be after check_in")
        if 'check_in' in values and (v - values['check_in']) > timedelta(days=90):
            raise ValueError("Stay length cannot exceed 90 days")
        return v


async def execute_tool(self, tool_call):
    if tool_call.name == "search_hotels":
        try:
            args = SearchHotelsArgs(**tool_call.arguments)
        except ValidationError as e:
            return {
                "status": "validation_error",
                "errors": e.errors(),
                "user_facing_hint": (
                    "Some search parameters were invalid. Confirm with the user before retrying."
                ),
                "retryable_after_correction": True,
            }
        return await self._search_hotels(args)

This catches three classes of bad calls:

Type errors: the LLM passes a string where the tool expects an integer.
Range errors: the LLM tries to search for 50 guests in one room.
Logical errors: check-out before check-in, dates in the past.

By rejecting these at the boundary with structured error responses, we prevent the deeper failure mode where the supplier API gets called with bad data, returns a cryptic error, the LLM cannot interpret the error, and a loop starts.

The Pydantic-based approach also gives you JSON schema generation for free, which feeds directly into the tool definitions you send to the LLM. Schema-aligned validation across both ends.

What I would not recommend

A few approaches we tried and abandoned:

Asking the LLM "are you done?" prompts mid-loop. Slows everything down and only works inconsistently. The orchestrator-level call budget is more reliable.
Letting the LLM see the full call history in every iteration. Increases context cost dramatically and provides little benefit. Pattern 4 (deduplication with structured feedback) is more efficient.
Streaming tool execution with partial results. Looks attractive but creates new failure modes where the LLM acts on incomplete data. Stick with atomic tool calls that either complete or fail cleanly.
Auto-generating tool definitions from API specs. Tempting because it sounds DRY, but auto-generated descriptions are usually not what the LLM needs. Hand-written tool descriptions, with explicit guidance about when to use the tool and when not to, work better.

Production results

After implementing these five patterns across our LLM-powered booking systems:

Agent loop incidents: dropped from 3 to 5 per week to under 1 per month.
Average tool calls per conversation: dropped 22%, mostly by eliminating duplicates and unnecessary retries.
Time-to-final-response: improved 18%, primarily from earlier short-circuiting of bad parameter calls.
Cost per conversation: dropped 31%, combination of fewer tool calls and tighter budget enforcement.

The patterns are not glamorous. They are mostly defensive engineering. But the alternative is the Claude Code incident: a $16,000 to $50,000 loss from an agent that "did not crash" but kept spending. In production LLM systems, the difference between cost-stable and not is exactly this kind of unsexy infrastructure.

If you are designing tools for an LLM agent, treat the tool interface as a contract that must hold even when the model is confused. The model will be confused. The contract will be tested. Patterns 1 through 5 are how it survives.

These patterns were developed across production builds at Adamo Software, including AI travel assistant deployments and agentic AI systems where tool-use reliability is non-negotiable.

How we reduced LLM hallucination to under 1% in a production booking system

Adamo Software — Tue, 28 Apr 2026 08:27:45 +0000

LLM hallucinations are not a research curiosity. They are a production problem. A 2026 Stanford AI Index reported hallucination rates across 26 leading LLMs ranging from 22% to 94%, and even the best-performing model on Vectara's leaderboard (Gemini-2.0-Flash-001) still hallucinates 0.7% of the time on grounded summarization tasks. In a booking system that handles thousands of transactions daily, even a 1% hallucination rate means dozens of bookings made with wrong prices, wrong dates, or wrong policies every single day.

This article walks through the validation pipeline we built that brought our production hallucination rate from approximately 4% (out of the box) to under 1%, and the architectural decisions that mattered most.

What "hallucination" means in a booking context

Before we get into the solution, it helps to be precise about what we are fixing. In a booking system, hallucinations break down into four categories:

Numerical hallucinations. The LLM states a price, a duration, a distance, or a count that does not match the source data. Example: the API returns $189/night, but the LLM tells the user "$179 per night." This is the most common and most damaging category.

Temporal hallucinations. Wrong times, wrong dates, wrong durations. Example: the API returns "departure 22:30 local time" and the LLM says "departure at 10:30 PM" (correct in 12-hour format) or "departure at 2:30 AM" (incorrect). The first is fine. The second is a hallucination.

Attribute hallucinations. The LLM invents amenities, policies, or features that are not in the source data. Example: claiming a hotel has a rooftop pool when the API returned no pool data, or stating a flight allows free cabin baggage when the fare class actually charges.

Citation hallucinations. The LLM references reviews, ratings, or sources that do not exist. Example: "According to recent guest reviews, the hotel scored 9.2 for cleanliness" when no review API was queried.

These categories matter because each requires a different detection strategy. A single validation pipeline that treats them uniformly will miss most of the failures.

The architecture: separating reasoning from facts

The core principle: the LLM never generates facts. It only formats them.

                ┌─────────────────────┐
   User query   │   LLM (reasoning)   │
   ───────────▶ │   Intent + Tool     │
                │   selection only    │
                └──────────┬──────────┘
                           │
                           ▼
                ┌─────────────────────┐
                │  Tool execution     │
                │  (real APIs)        │
                │  ───────────────    │
                │  • search_hotels    │
                │  • get_pricing      │
                │  • check_avail      │
                └──────────┬──────────┘
                           │
                           ▼
                ┌─────────────────────┐
                │  Structured data    │
                │  (canonical facts)  │
                └──────────┬──────────┘
                           │
                           ▼
                ┌─────────────────────┐
                │   LLM (formatter)   │
                │   Generates prose   │
                │   from facts only   │
                └──────────┬──────────┘
                           │
                           ▼
                ┌─────────────────────┐
                │ Validation pipeline │
                │ Checks LLM output   │
                │ against facts       │
                └──────────┬──────────┘
                           │
                           ▼
                       User reply

The LLM has two roles, executed in two separate calls:

Reasoning call. Given the user's message, decide which tools to invoke and with what parameters. Output: a list of tool calls. No facts generated yet.
Formatting call. Given the tool results (structured JSON), generate a natural language response. The system prompt explicitly forbids introducing any numerical, temporal, or factual claim not present in the tool output.

This separation alone reduced our hallucination rate from ~4% to ~2%. The remaining 2% came from the LLM still occasionally introducing claims during the formatting step. That is what the validation pipeline catches.

The validation pipeline

The pipeline runs every LLM-formatted response through three checks before sending it to the user.

Check 1: Numerical claim extraction and verification

Every number in the LLM's response must trace back to the structured tool output. We extract numbers using regex (with currency, time, and unit awareness), then verify each one against the canonical fact set.

import re
from decimal import Decimal

# Patterns for different numerical claim types
PRICE_PATTERN = re.compile(
    r'\$\s?(\d{1,3}(?:,\d{3})*(?:\.\d{1,2})?)|(\d+(?:\.\d{1,2})?)\s?(?:USD|dollars?)',
    re.IGNORECASE
)
TIME_PATTERN = re.compile(
    r'\b(\d{1,2}):(\d{2})\s?(AM|PM|am|pm)?\b'
)
PERCENT_PATTERN = re.compile(r'(\d+(?:\.\d+)?)\s?%')

def extract_numerical_claims(text: str) -> dict:
    """Extract all numerical claims from LLM-generated text."""
    claims = {
        'prices': [],
        'times': [],
        'percentages': [],
    }

    for match in PRICE_PATTERN.finditer(text):
        value = match.group(1) or match.group(2)
        claims['prices'].append(Decimal(value.replace(',', '')))

    for match in TIME_PATTERN.finditer(text):
        hour, minute, meridiem = match.groups()
        claims['times'].append({
            'raw': match.group(0),
            'hour': int(hour),
            'minute': int(minute),
            'meridiem': meridiem,
        })

    for match in PERCENT_PATTERN.finditer(text):
        claims['percentages'].append(Decimal(match.group(1)))

    return claims


def validate_claims(claims: dict, canonical_facts: dict) -> list:
    """Return list of unsupported claims."""
    violations = []

    # Build lookup sets from canonical facts
    valid_prices = {Decimal(str(p)) for p in canonical_facts.get('prices', [])}
    valid_times = canonical_facts.get('times', [])

    for price in claims['prices']:
        if price not in valid_prices:
            # Allow small rounding (±$0.01)
            if not any(abs(price - vp) <= Decimal('0.01') for vp in valid_prices):
                violations.append(('price', price, valid_prices))

    for time_claim in claims['times']:
        if not is_time_in_canonical(time_claim, valid_times):
            violations.append(('time', time_claim, valid_times))

    return violations

If any claim fails verification, the response is rejected and the LLM is asked to regenerate with stricter instructions. After two failed attempts, the system falls back to a templated response built directly from the structured data.

Check 2: Attribute grounding via embedding similarity

Numerical extraction does not catch attribute hallucinations ("the hotel has a rooftop pool"). For these, we use a different approach: embedding-based grounding.

For every property the LLM mentions in its response, we have a canonical attribute list from the API. We compute embeddings for both the LLM's claims (extracted as noun phrases) and the canonical attributes, then check that every claim has a high-similarity match in the canonical set.

from sentence_transformers import SentenceTransformer
import numpy as np

embedder = SentenceTransformer('all-MiniLM-L6-v2')

def extract_noun_phrases(text: str) -> list:
    """Extract attribute claims using a lightweight NLP pipeline."""
    # Simplified: use spaCy noun chunks in production
    import spacy
    nlp = spacy.load('en_core_web_sm')
    doc = nlp(text)
    return [chunk.text for chunk in doc.noun_chunks]


def validate_attributes(
    llm_text: str,
    canonical_attributes: list[str],
    threshold: float = 0.7
) -> list:
    """Flag attributes claimed in LLM text that don't match canonical data."""
    claimed_phrases = extract_noun_phrases(llm_text)

    if not claimed_phrases or not canonical_attributes:
        return []

    claim_embeddings = embedder.encode(claimed_phrases)
    canonical_embeddings = embedder.encode(canonical_attributes)

    violations = []
    for phrase, claim_emb in zip(claimed_phrases, claim_embeddings):
        similarities = np.dot(canonical_embeddings, claim_emb) / (
            np.linalg.norm(canonical_embeddings, axis=1) *
            np.linalg.norm(claim_emb)
        )
        if similarities.max() < threshold:
            violations.append({
                'claim': phrase,
                'best_match_score': float(similarities.max()),
                'best_match': canonical_attributes[similarities.argmax()],
            })

    return violations

This catches statements like "the hotel offers a fitness center and spa" when the API only returned "fitness center" (no spa). The threshold of 0.7 was tuned empirically. Lower thresholds let too many attribute hallucinations through. Higher thresholds caused false positives on legitimate paraphrases.

Check 3: Self-consistency via second LLM pass

For complex multi-sentence responses, regex and embeddings miss subtle errors. The third check uses a second LLM call (with a smaller, cheaper model) configured as a fact-checker.

FACT_CHECK_PROMPT = """You are a strict fact-checker. Given:
1. A SOURCE DATA object (the only ground truth)
2. A CANDIDATE RESPONSE (what we want to send to a user)

Your job: identify any claim in the CANDIDATE that is NOT supported by the SOURCE.

Output JSON only:
{
  "supported": true/false,
  "violations": [
    {"claim": "exact text from candidate", "reason": "why not supported"}
  ]
}

Do not flag stylistic differences or paraphrasing. Only flag factual claims that contradict or are not present in SOURCE.

SOURCE DATA:
{source_json}

CANDIDATE RESPONSE:
{response_text}
"""


async def fact_check(response_text: str, source_data: dict) -> dict:
    result = await call_llm(
        model='gpt-4o-mini',  # cheaper model for fact-checking
        prompt=FACT_CHECK_PROMPT.format(
            source_json=json.dumps(source_data, indent=2),
            response_text=response_text,
        ),
        response_format={'type': 'json_object'},
        temperature=0,  # deterministic fact-checking
    )
    return json.loads(result)

This catches hallucinations that are linguistically subtle. Example: the source data says "free cancellation until 24 hours before check-in" and the LLM writes "you can cancel free of charge anytime before your stay." The numerical extractor sees nothing wrong (no numbers conflict). The embedding check thinks it is similar enough. But the fact-checker catches that "anytime" contradicts "until 24 hours before."

The fact-checker call adds 200 to 400ms of latency. We run it asynchronously and only block the user-facing response when the fact-checker flags violations.

Results from production

After 60 days of running this pipeline in production:

Numerical hallucinations: dropped from ~2.4% to 0.3%
Temporal hallucinations: dropped from ~0.8% to 0.1%
Attribute hallucinations: dropped from ~1.2% to 0.4%
Citation hallucinations: dropped to 0% (we eliminated review/rating citations from the formatter prompt entirely)

Combined hallucination rate: under 1% across all conversation turns. The 0.8% to 0.9% that remains is dominated by edge cases in attribute paraphrasing (e.g., the LLM saying "ocean view" when the source said "sea view"). These are technically inaccurate but rarely cause booking errors.

Things that did not work

A few approaches we tried and abandoned:

Higher temperature with self-correction prompts. Asking the LLM "are you sure?" or "double-check your facts" before responding reduced hallucinations slightly but was inconsistent. Google's December 2024 research found that asking "are you hallucinating right now?" reduces hallucination rates by 17%, but the effect diminishes after 5-7 interactions. We confirmed this empirically.
Fine-tuning on our domain data. Improved phrasing and tone but did not reliably reduce hallucinations. The model still invented prices and times confidently. Fine-tuning addresses style, not factual grounding.
Constraining output to JSON-only. Useful for tool calls but unsuitable for user-facing responses. Travelers want natural language, not structured data.
Reasoning models for everything. Paradoxically, OpenAI's o3 reasoning model hallucinated 33% of the time on PersonQA, double the rate of its predecessor o1, and the smaller o4-mini performed even worse at 48%. Reasoning models excel at analysis but introduce more hallucinations on factual tasks. We use reasoning mode only for intent parsing, never for response generation.

Practical takeaways

If you are building any LLM-powered system where factual accuracy matters (booking, customer service, financial advisory, healthcare assistance), the architectural rules that mattered most for us:

Two LLM calls, not one. Reasoning and formatting are different jobs. Mixing them in a single call lets the LLM hallucinate facts while reasoning.
Structured ground truth as the only fact source. If a claim is not in the structured tool output, it cannot be in the response. Period.
Validation as a hard gate. Logging hallucinations is not enough. Block them from reaching the user, even at the cost of latency and occasional templated responses.
Layer your detection. No single check (regex, embeddings, fact-checker) catches everything. Layered checks with different signal types catch different failure modes.
Measure rates by category. "Hallucination rate" as a single number is not actionable. Numerical, temporal, attribute, and citation hallucinations have different costs and different solutions.

The fundamental insight is that hallucination is not a model problem. It is a system design problem. The Stanford AI Index 2026 calls for engineers to "treat hallucination rates as design inputs rather than bugs to be ignored". That has been our experience exactly. Pick a baseline model with reasonable performance, then build the pipeline around it that enforces grounding.

This pattern was developed across several production builds at Adamo Software, including AI travel assistants and AI chatbots for travel booking where booking accuracy is non-negotiable.

How we handle LLM context window limits without losing conversation quality

Adamo Software — Tue, 21 Apr 2026 08:17:52 +0000

Every developer building on LLMs hits the same wall eventually. Your chatbot works beautifully for the first 10 turns, then starts forgetting things. Your agent ran a 30-step workflow and lost track of the original goal halfway through. Your RAG system stuffed so much context into the prompt that response quality dropped.

This is the context window problem, and it does not go away by switching to a model with a bigger window. We learned this the hard way while building an AI assistant for a travel booking platform. This post covers the strategies we actually use in production, with the trade-offs we hit.

Why bigger context windows are not the answer

Claude 3.5 Sonnet has a 200K token window. GPT-4o has 128K. Gemini 1.5 Pro has up to 2M. The temptation is to just throw everything in.

Three problems with that approach.

First, cost. Input tokens are not free. At 2M tokens per call, you are spending significant money on every request even before the model generates anything.

Second, latency. Processing a 200K-token prompt takes meaningfully longer than a 10K-token one. For a chat interface, this is the difference between instant and sluggish.

Third, and most importantly, quality degrades with length. Research from Anthropic and others has consistently shown that models pay less attention to content in the middle of very long contexts. This is called the "lost in the middle" problem. A fact placed at token 80,000 of a 150,000-token context has a real chance of being ignored.

So the question is not "how do we fit everything," it is "what actually needs to be in the prompt right now."

The four strategies we use

We combine four techniques depending on the use case. None of these are novel individually. The value is in knowing when to use which.

1. Sliding window with summarization

For chatbots and conversational agents, we keep the last N turns verbatim and summarize everything older. The key design decision is how often to summarize.

from typing import List
from dataclasses import dataclass

@dataclass
class Message:
    role: str
    content: str
    tokens: int

RECENT_TURNS = 6
SUMMARIZE_THRESHOLD = 20

def manage_context(messages: List[Message], summary: str) -> tuple[List[Message], str]:
    if len(messages) <= SUMMARIZE_THRESHOLD:
        return messages, summary

    # Keep the last N turns raw
    recent = messages[-RECENT_TURNS:]
    to_summarize = messages[:-RECENT_TURNS]

    # Incremental summarization: feed old summary + new messages
    new_summary = summarize(
        existing_summary=summary,
        new_messages=to_summarize
    )
    return recent, new_summary

We trigger summarization when the conversation exceeds 20 turns, not on every turn. Summarizing every turn is wasteful and introduces quality drift because you are summarizing summaries of summaries.

The trade-off: summaries lose specificity. If a user mentioned "I prefer aisle seats near the front" on turn 3 and you compressed that into "user discussed seat preferences" on turn 25, the agent may forget the actual preference. We mitigate this with strategy #3 below.

2. Relevance-based retrieval instead of full history

For long-running agents that make many tool calls, we do not send the entire tool call history back on every step. Instead, we embed each prior action and its result, and retrieve only the top-k most relevant to the current step.

def build_agent_context(current_goal: str, all_steps: List[Step], k: int = 5):
    # Embed the current goal
    query_embedding = embed(current_goal)

    # Embed each step's summary
    step_embeddings = [embed(f"{s.action}: {s.result}") for s in all_steps]

    # Retrieve top-k most relevant prior steps
    scores = cosine_similarity(query_embedding, step_embeddings)
    top_k_indices = np.argsort(scores)[-k:]
    relevant_steps = [all_steps[i] for i in sorted(top_k_indices)]

    return relevant_steps

This works well when agent steps are semantically diverse. It works poorly when every step is similar, because the embeddings cluster too tightly. For those cases we fall back to the sliding window.

3. Structured memory for facts that must not be lost

Some information cannot be lost to summarization. User preferences, confirmed bookings, authentication context, critical constraints. We extract these into a structured memory object that travels with every prompt.

structured_memory = {
    "user_profile": {
        "name": "extracted_from_conversation",
        "preferences": ["aisle seat", "non-smoking", "high floor"],
    },
    "session_state": {
        "current_booking": {"destination": "Tokyo", "dates": "2026-06-12 to 2026-06-20"},
        "confirmed_steps": ["flight_selected", "hotel_searched"],
    },
    "hard_constraints": ["budget: $3000 max", "must arrive before June 14"]
}

The LLM does not write to this object freely. We use a dedicated extraction step after each turn, with a structured output schema, to pull out facts. This gives us deterministic memory instead of relying on the model to remember.

The Anthropic prompt caching documentation is worth reading if you go this route, because a stable memory block at the start of your prompt is an ideal cache target.

4. Context compression for large retrieved documents

For RAG systems retrieving long documents, we compress before injection. Instead of pasting a 5000-word document into the context, we run a fast model (Haiku or GPT-4o-mini) to extract only the passages relevant to the user's query.

This is a two-model pipeline:

Retrieval returns top-k documents (often 3-5 long docs)
A fast, cheap model extracts relevant sections from each
The main model sees only the compressed, relevant content

The extra inference call adds ~200ms of latency but typically reduces main prompt size by 70-85%. Net cost is lower and quality is usually higher because the main model is not distracted by irrelevant content.

When each strategy fails

Being specific about failure modes, because this is where blog posts usually wave their hands:

Sliding window fails when users reference something from far back in the conversation ("like that restaurant I mentioned earlier"). Always pair with structured memory.
Relevance retrieval fails when the current step has no good semantic overlap with prior relevant steps. For example, if step 30 needs information from step 2 but they use completely different vocabulary, retrieval misses it.
Structured memory fails when the extraction step produces low-quality outputs. Garbage in, garbage out. We validate extractions against a Pydantic schema and retry with a stricter prompt on validation failure.
Context compression fails when the query is ambiguous. If the user asks "tell me more about that," the compression model has no way to know what "that" refers to. We rewrite the query using recent conversation context before passing it to compression.

What changed when we combined all four

Before we had a structured context strategy, a 50-turn conversation in our travel agent would produce noticeably worse responses by turn 40. Users would need to re-state preferences. The agent would propose options the user had already rejected.

After combining sliding window + relevance retrieval + structured memory:

Average tokens per request dropped from ~18,000 to ~6,500, a 64% reduction
User-reported "the AI forgot what I said" complaints dropped significantly in internal testing
Response latency p95 improved from 4.2s to 2.1s

One thing we did not improve: cost per successful conversation. The reduction in tokens was offset by the extra inference calls for summarization and extraction. What we got was better quality at roughly the same cost, which for a production agent is the right trade.

Wrapping up

The context window is a constraint to design around, not a capacity to fill. A model with 2M tokens gives you more runway, but if you depend on stuffing everything in, your quality will still degrade and your costs will still climb.

Start with a sliding window for recent turns, structured memory for facts that matter, and retrieval for everything in between. Compression is the advanced move once the basics are in place.

If you are working on production AI systems and want deeper context on multi-step agent design, we have written previously about AI agent fallback chains and human-in-the-loop patterns that pairs well with this post. For background reading, Greg Kamradt's Needle in a Haystack benchmarks are a good way to see context window degradation empirically.

I work on AI platform engineering at Adamo Software, where we build custom AI systems for travel, healthcare, and enterprise clients.

Why travel search is harder than eCommerce search: The technical differences most developer miss

Adamo Software — Tue, 14 Apr 2026 02:54:30 +0000

If you have built search for an e-commerce product and then moved to a travel project, you know the feeling. Everything you assumed about search breaks within the first week. The data model is different. The freshness requirements are different. The query patterns are different. Even the definition of "in stock" is different.

This is not a difficulty ranking. E-commerce search has its own hard problems. But the two domains diverge in ways that catch experienced developers off guard, and understanding those differences before you start building saves weeks of rework.

Inventory is perishable, not stockable

In e-commerce, a product either exists in the warehouse or it does not. A pair of shoes in size 42 is available until someone buys the last pair. The inventory state changes when a purchase happens. Between purchases, the state is stable. You can cache product availability for minutes or even hours without causing problems.

In travel, inventory expires whether anyone buys it or not. A hotel room on April 15th ceases to exist on April 16th. A flight seat for the 9am departure is worthless at 9:01am. This means every search result has a built-in expiration timestamp that has nothing to do with purchase activity.

The technical consequence: your caching strategy needs to account for time-based expiration, not just event-based invalidation. A hotel room that was available 30 minutes ago might still be available (nobody booked it) or might be gone (someone booked it on another channel). You cannot know without checking the source system. E-commerce developers used to comfortable cache TTLs discover that travel search requires either very short TTLs or real-time availability checks on every result, both of which have cost and latency implications.

One product, many sources of truth

In e-commerce, your product catalog is your source of truth. You control it. When you update a price or mark something out of stock, the change propagates through your system. There is one database, one version of the truth.

In travel, the same hotel room is sold simultaneously through the hotel's own website, Booking.com, Expedia, Agoda, and three other OTAs. Each channel has its own cached version of availability. The hotel's Property Management System (PMS) is the theoretical source of truth, but updates to the PMS propagate to each channel at different speeds through different APIs.

When a developer queries availability for "hotels in Da Nang, July 15-18," the response depends on which system they queried, when they queried it, and how recently that system synced with the PMS. Two queries made 30 seconds apart can return different results, not because availability changed, but because the cache refresh cycle hit between them.

In e-commerce, if your search returns a product, the customer can almost certainly buy it. In travel, if your search returns a room, the customer might click through and discover it was booked on another channel 45 seconds ago. This is why travel platforms need a confirmation step that re-checks availability at the moment of booking, a pattern that e-commerce checkout rarely requires.

Price is a function, not a field

In e-commerce, a product has a price. It might change during a sale or promotion, but at any given moment, the price is a stored value in a database column. You query it, you display it.

In travel, price is computed at query time based on a combination of factors: the dates selected, the number of guests, the room type, the cancellation policy chosen, the customer's loyalty tier, the time of day, demand levels, competitor pricing, and sometimes even the customer's country of origin (due to regional pricing agreements).

A single hotel room does not have "a price." It has a pricing function that returns different values depending on the parameters. This means travel search cannot simply index prices in Elasticsearch the way e-commerce search indexes product prices. You either pre-compute prices for common date and occupancy combinations (expensive in storage, complex to keep fresh) or you compute prices at search time by calling the supplier's pricing API (expensive in latency, subject to rate limits).

Most travel search systems use a hybrid: pre-computed base rates for display in search results, with real-time pricing API calls when the user clicks through to a specific property. The mismatch between these two numbers is a constant source of user frustration ("the price changed when I clicked") and engineering headaches.

Queries are multi-dimensional by default

An e-commerce search query is typically a keyword with optional filters. "Running shoes, size 42, black, under $100." The search engine matches keywords against product attributes and applies filters. The core operation is text matching plus filtering.

A travel search query always involves at least three dimensions that interact with each other: location, dates, and occupancy. "Hotels in Hoi An, July 15-18, 2 adults 1 child." These three dimensions are not independent filters. The location determines which properties exist. The dates determine which of those properties have availability. The occupancy determines which room types within available properties can accommodate the guests.

In e-commerce, you can filter sequentially: find all shoes, then filter by size, then by color. In travel, you need to evaluate all three dimensions simultaneously because a property might have availability for 2 adults on July 15-17 but not July 17-18, or it might have a room for 2 adults but not 2 adults plus 1 child on those specific dates.

This multi-dimensional constraint satisfaction is why travel search is computationally heavier than e-commerce search at the query level, and why GDS (Global Distribution System) APIs are notoriously slow. They are doing constraint satisfaction across millions of inventory records, not keyword matching.

Sorting is subjective and context-dependent

E-commerce search has well-understood sorting defaults: relevance, price low to high, price high to low, bestselling, newest. These are straightforward to implement because the sorting criteria are attributes stored on the product.

Travel search sorting is more ambiguous. "Best" for a business traveler means close to the meeting location, with fast WiFi and a desk. "Best" for a family means close to the beach, with a pool and a kids' club. "Best" for a budget backpacker means cheapest per night with decent reviews. The same inventory, the same dates, completely different optimal orderings.

This is why personalization has a larger impact on conversion in travel search than in e-commerce search. In e-commerce, sorting by "bestselling" is a reasonable default that serves most users. In travel, there is no universal default that works. The platform either invests in personalized ranking or accepts that search results will feel generic to most users.

What this means if you are building travel search

If you are moving from e-commerce to travel development, these are the adjustments that will save you the most time:

Rethink your caching strategy. E-commerce caching patterns (cache for 5-15 minutes, invalidate on purchase events) do not transfer. Travel search needs either much shorter TTLs or a two-phase approach: cached results for browsing, real-time confirmation at booking time. Budget for higher infrastructure costs.

Separate "display price" from "booking price." Accept that the price shown in search results will sometimes differ from the price at checkout. Build the UX to handle this gracefully (price change notifications, rate locks) rather than trying to eliminate the mismatch entirely. Eliminating it is prohibitively expensive at scale.

Index availability windows, not availability states. Instead of a boolean "available: true/false," store date ranges with room type and occupancy constraints. Your search index needs to answer "is this property available for these specific dates and this specific guest configuration?" not just "is this property available?"

Plan for multi-source data reconciliation from day one. If you are aggregating inventory from multiple suppliers or OTAs, build a normalization layer that maps different data formats into a unified schema before it hits your search index. Do not let supplier-specific data structures leak into your search logic.

Build the confirmation step into your booking flow. Unlike e-commerce where "add to cart" is low-stakes, travel search results go stale fast. The availability check at booking time is not optional. Design your UX and your API around the assumption that some results will be unavailable by the time the user clicks "book."

Built by the engineering team at Adamo Software. We build custom platforms for travel, healthcare, and enterprise applications.

Building an AI fallback system: when to use GPT-4o, when to fall back to Haiku, when to skip the LLM entirely

Adamo Software — Tue, 07 Apr 2026 03:40:23 +0000

Not every query deserves a frontier model. A user asking "what is your cancellation policy?" does not need GPT-4o to generate the answer. A rules engine or a simple database lookup handles it in 5 milliseconds at zero token cost.

We learned this the hard way. Our first production deployment sent everything through GPT-4o. The quality was great. The bill was $7,200/month for a feature that should have cost $2,000. Worse, 60% of those queries were simple enough that a smaller model (or no model at all) would have produced identical output.

This article covers the three-tier fallback system we built: a rules engine for deterministic queries, a cheap model (Claude Haiku) for simple generation, and a frontier model (GPT-4o) for complex reasoning. Stack: Node.js 20, TypeScript.

The three tiers

Here is the routing logic:

Incoming query
    ↓
┌─────────────────────┐
│  Tier 0: Rules      │  → deterministic lookup, no LLM
│  (FAQ, status, data)│     cost: $0, latency: <10ms
└─────────┬───────────┘
          ↓ not matched
┌─────────────────────┐
│  Tier 1: Haiku      │  → simple generation
│  (summaries, format)│     cost: $1/$5 per 1M tokens
└─────────┬───────────┘
          ↓ quality check fails
┌─────────────────────┐
│  Tier 2: GPT-4o     │  → complex reasoning
│  (analysis, compare)│     cost: $2.50/$10 per 1M tokens
└─────────────────────┘

The classifier that decides the tier is itself a cheap LLM call. We use GPT-4o-mini with a one-line system prompt. The classification costs roughly $0.0001 per request, which is negligible.

Tier 0: skip the LLM entirely

This is the highest-ROI tier because it costs nothing. Before any query hits an LLM, we check if it matches a deterministic pattern:

interface TierZeroRule {
  patterns: RegExp[];
  handler: (query: string, context: any) => string;
}

const tierZeroRules: TierZeroRule[] = [
  {
    // FAQ lookups
    patterns: [
      /cancellation\s*policy/i,
      /refund\s*policy/i,
      /check.?in\s*time/i,
      /check.?out\s*time/i,
    ],
    handler: (query, context) => {
      return faqDatabase.findBestMatch(query);
    },
  },
  {
    // Booking status (pure data lookup)
    patterns: [
      /booking\s*(status|confirmation)/i,
      /order\s*#?\d+/i,
    ],
    handler: (query, context) => {
      const bookingId = extractBookingId(query);
      return bookingService.getStatus(bookingId);
    },
  },
  {
    // Price checks (structured data, no generation needed)
    patterns: [
      /how much.*(cost|price)/i,
      /price\s*(for|of)/i,
    ],
    handler: (query, context) => {
      return pricingService.lookup(query, context);
    },
  },
];

function tryTierZero(query: string, context: any): string | null {
  for (const rule of tierZeroRules) {
    if (rule.patterns.some(p => p.test(query))) {
      return rule.handler(query, context);
    }
  }
  return null; // no match, proceed to LLM tiers
}

In our system, Tier 0 catches 22% of all queries. Those are 22% of queries that never touch an LLM, never cost a token, and return in under 10 milliseconds.

The key insight: do not overthink the pattern matching. Simple regex works. If a query contains "cancellation policy", you know the answer. You do not need embeddings or a classifier for this.

The classifier: deciding between Tier 1 and Tier 2

For queries that pass Tier 0, we need to decide: cheap model or expensive model? We use a lightweight classifier:

async function classifyComplexity(query: string): Promise<'simple' | 'complex'> {
  const response = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      {
        role: 'system',
        content: `Classify the user query as SIMPLE or COMPLEX.
SIMPLE: factual questions, summaries, formatting, translation, single-step tasks.
COMPLEX: comparisons, multi-step reasoning, analysis, recommendations with tradeoffs, ambiguous intent.
Reply with one word only.`,
      },
      { role: 'user', content: query },
    ],
    max_tokens: 1,
    temperature: 0,
  });

  const classification = response.choices[0].message.content?.trim().toUpperCase();
  return classification === 'COMPLEX' ? 'complex' : 'simple';
}

This adds about 200ms latency and costs $0.0001 per call. At 10,000 queries/day, the classifier costs $1/day total. The savings from routing 60% of queries to Haiku instead of GPT-4o are roughly $150/day.

Tier 1 and Tier 2: the fallback chain

Here is where the fallback logic lives. We call Tier 1 first. If the response fails a quality check, we escalate to Tier 2:

async function processQuery(query: string, context: any) {
  // Tier 0: deterministic
  const tierZeroResult = tryTierZero(query, context);
  if (tierZeroResult) {
    return { response: tierZeroResult, tier: 0, cost: 0 };
  }

  // Classify
  const complexity = await classifyComplexity(query);

  if (complexity === 'simple') {
    // Tier 1: Haiku
    const haiku = await callModel('claude-haiku', query, context);

    if (passesQualityCheck(haiku, query)) {
      return { response: haiku, tier: 1, cost: haiku.tokenCost };
    }

    // Fallback to Tier 2 if Haiku output is low quality
    const gpt4o = await callModel('gpt-4o', query, context);
    return { response: gpt4o, tier: 2, cost: haiku.tokenCost + gpt4o.tokenCost };
  }

  // Complex queries go straight to Tier 2
  const gpt4o = await callModel('gpt-4o', query, context);
  return { response: gpt4o, tier: 2, cost: gpt4o.tokenCost };
}

The quality check is the most important function in the entire system. A bad quality check either wastes money (escalating when unnecessary) or serves bad responses (not escalating when needed).

Our quality check is simple:

function passesQualityCheck(response: ModelResponse, query: string): boolean {
  // Check 1: response is not empty or too short
  if (!response.text || response.text.length < 20) return false;

  // Check 2: model did not refuse or hedge excessively
  const hedgePatterns = [
    /i'm not sure/i,
    /i don't have.*information/i,
    /i cannot.*determine/i,
  ];
  if (hedgePatterns.some(p => p.test(response.text))) return false;

  // Check 3: response addresses the query topic
  // (simple keyword overlap check, not semantic)
  const queryKeywords = extractKeywords(query);
  const responseKeywords = extractKeywords(response.text);
  const overlap = queryKeywords.filter(k => responseKeywords.includes(k));
  if (overlap.length < queryKeywords.length * 0.3) return false;

  return true;
}

We intentionally kept this rule-based, not LLM-based. Using another LLM call to check quality would add latency and cost that defeats the purpose.

Results after 6 weeks

Metric	Before (GPT-4o only)	After (3-tier)
Monthly LLM cost	$7,200	$2,100
Avg response latency	1.8s	0.6s
Tier 0 (no LLM)	0%	22%
Tier 1 (Haiku)	0%	51%
Tier 2 (GPT-4o)	100%	27%
Fallback rate (Tier 1→2)	n/a	4.2%
User satisfaction (CSAT)	4.3/5	4.2/5

The CSAT dropped by 0.1 points. We consider that acceptable for a 71% cost reduction. The drop came entirely from the 4.2% of queries where Haiku's response was served but was slightly less thorough than what GPT-4o would have produced. The quality check catches the obvious failures, but subtle quality differences slip through.

What we would do differently

Track cost per tier from day one. We only started logging which tier handled each query after week 2. Those first two weeks of data were lost, making it harder to benchmark improvements.

Start with a stricter quality check and loosen it. Our first quality check was too lenient. It let through some Haiku responses that should have been escalated. We tightened the keyword overlap threshold from 0.2 to 0.3, which increased the fallback rate from 2.1% to 4.2% but eliminated the worst-quality responses.

Consider a Tier 1.5. We now think there is room for a middle tier between Haiku ($1/$5 per 1M tokens) and GPT-4o ($2.50/$10). Something like GPT-4o-mini or Claude Sonnet for queries that are too complex for Haiku but do not need frontier-level reasoning. We are testing this now.

Wrapping up

The single biggest cost optimization in our LLM stack was not caching, not prompt compression, not fine-tuning. It was routing queries to the right tier. 22% of queries never needed an LLM. 51% needed only a cheap model. Only 27% actually required GPT-4o.

If you are running everything through a single frontier model, start by logging your queries for a week and categorizing them by complexity. You will likely find that the majority do not need frontier reasoning. Build Tier 0 first (it is free), add a classifier, and let the data tell you where the boundaries should be.

Built by the engineering team at Adamo Software. We build AI-powered platforms for travel, healthcare, and enterprise applications.

Building a real-time travel search engine: lessons from integrating with GDS APIs

Adamo Software — Mon, 30 Mar 2026 06:48:09 +0000

If you have ever tried to integrate with a Global Distribution System API (Amadeus, Sabre, or Travelport), you know it is not like calling a typical REST endpoint. GDS APIs were architected decades ago, carry SOAP/XML legacy patterns even in their modern REST wrappers, and enforce aggressive rate limits that can break your search experience during a traffic spike. This article shares what we learned building a travel search service that queries multiple GDS providers in parallel, normalizes their responses into a unified schema, and caches results intelligently to stay within quota limits.

The project context

We were building a custom booking platform for a mid-size tour operator. The core requirement: let travelers search flights, hotels, and packages from a single search bar, with results aggregated from Amadeus Self-Service APIs, a Sabre REST endpoint, and two direct hotel suppliers via proprietary APIs. The stack was Node.js (Fastify) for the search service, Redis for caching, and Elasticsearch for indexing normalized inventory.

The naive approach would be to call each supplier sequentially, merge results, and return them. That gives you 3 to 5 second response times. Travelers leave after 2 seconds. So the real engineering challenge was not "can we connect to a GDS" but "can we make multi-supplier search feel instant."

Authentication: the part nobody warns you about

Every GDS has its own auth flow, and they all have quirks.

Amadeus Self-Service uses OAuth 2.0 with client credentials. You get an access token that expires in 30 minutes. Straightforward, except the token refresh has a subtle gotcha: if you refresh too aggressively under load, you can hit a secondary rate limit on the auth endpoint itself. We solved this by implementing a singleton token manager that refreshes proactively at the 25-minute mark, not on expiry.

// Token manager with proactive refresh
class AmadeusTokenManager {
  constructor(clientId, clientSecret) {
    this.clientId = clientId;
    this.clientSecret = clientSecret;
    this.token = null;
    this.expiresAt = 0;
  }

  async getToken() {
    // Refresh 5 minutes before expiry
    if (Date.now() < this.expiresAt - 300000) {
      return this.token;
    }

    const res = await fetch(
      'https://api.amadeus.com/v1/security/oauth2/token',
      {
        method: 'POST',
        headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
        body: new URLSearchParams({
          grant_type: 'client_credentials',
          client_id: this.clientId,
          client_secret: this.clientSecret,
        }),
      }
    );

    const data = await res.json();
    this.token = data.access_token;
    // Amadeus tokens last 1799 seconds (~30 min)
    this.expiresAt = Date.now() + data.expires_in * 1000;
    return this.token;
  }
}

Sabre uses a similar OAuth flow but requires a base64-encoded composite of client ID and secret in the authorization header, not in the body. It is a small difference that costs you an hour of debugging if you miss it.

The lesson: do not assume GDS auth flows are interchangeable. Read each provider's docs carefully, and build a provider-specific auth module from the start.

Parallel search with timeout control

The search service fires requests to all suppliers simultaneously using Promise.allSettled. This is critical. Promise.all would fail the entire search if one supplier times out. With allSettled, we return whatever results come back within the timeout window.

async function searchAllSuppliers(query, timeoutMs = 2000) {
  const suppliers = [
    searchAmadeus(query),
    searchSabre(query),
    searchDirectHotelA(query),
    searchDirectHotelB(query),
  ];

  // Wrap each with a timeout
  const withTimeout = suppliers.map((p, i) =>
    Promise.race([
      p.then(res => ({ supplier: i, status: 'ok', data: res })),
      new Promise(resolve =>
        setTimeout(
          () => resolve({ supplier: i, status: 'timeout', data: [] }),
          timeoutMs
        )
      ),
    ])
  );

  const results = await Promise.allSettled(withTimeout);

  return results
    .filter(r => r.status === 'fulfilled' && r.value.status === 'ok')
    .flatMap(r => r.value.data);
}

We set the timeout at 2000ms. If Amadeus responds in 800ms and Sabre takes 3 seconds, the user sees Amadeus results immediately. We log the Sabre timeout and investigate later. In practice, Amadeus Self-Service APIs responded in 400 to 1200ms for flight searches. Sabre was more variable, ranging from 300ms to 2500ms depending on route complexity.

The caching strategy that saved our quota

This is where most GDS integrations either succeed or blow their budget. Amadeus Self-Service provides a free monthly quota per API, with per-call charges of roughly $0.001 to $0.025 once you exceed it (Amadeus Pricing). At 10,000 searches per day, you exhaust free quotas fast.

Our caching strategy has three layers:

Layer 1: Exact-match cache (Redis, TTL 5 minutes). Same origin, destination, dates, passengers. This catches repeated searches from the same user session and from multiple users searching popular routes.

function buildCacheKey(query) {
  const { origin, destination, departDate, returnDate, pax } = query;
  return `search:${origin}:${destination}:${departDate}:${returnDate || 'oneway'}:${pax}`;
}

async function cachedSearch(query) {
  const key = buildCacheKey(query);
  const cached = await redis.get(key);

  if (cached) {
    return JSON.parse(cached);
  }

  const results = await searchAllSuppliers(query);
  // Cache for 5 minutes. Flight prices change,
  // but not every 30 seconds.
  await redis.setex(key, 300, JSON.stringify(results));
  return results;
}

Layer 2: Fuzzy-match cache (Redis, TTL 15 minutes). If a user searches HAN to NRT on June 15, and another searches HAN to NRT on June 14 or 16, the price structure is usually similar. We serve the cached result with a "prices may vary" indicator while firing a background refresh. This cut our API calls by roughly 40%.

Layer 3: Prewarming popular routes (cron job, every 30 minutes). We identified the top 50 route pairs from booking history and pre-fetched their availability on a schedule. This means the first search of the day for a popular route hits cache, not the GDS.

Combined, these three layers reduced our actual GDS API calls by approximately 65%, keeping us well within free quotas during the first months of operation.

Data normalization: the unglamorous but essential part

Each GDS returns data in wildly different formats. Amadeus returns a nested JSON structure with dictionaries for airline and location codes. Sabre returns a flatter structure but uses different field names and embeds fare rules differently. Direct hotel APIs return whatever the hotel decided to put in their XML feed.

We built a normalization layer with a simple interface:

// Each supplier implements this interface
function normalizeAmadeusFlightOffer(raw) {
  return {
    id: `amadeus_${raw.id}`,
    supplier: 'amadeus',
    type: 'flight',
    origin: raw.itineraries[0].segments[0].departure.iataCode,
    destination: raw.itineraries[0].segments.at(-1).arrival.iataCode,
    departureTime: raw.itineraries[0].segments[0].departure.at,
    arrivalTime: raw.itineraries[0].segments.at(-1).arrival.at,
    stops: raw.itineraries[0].segments.length - 1,
    airline: raw.validatingAirlineCodes[0],
    price: {
      amount: parseFloat(raw.price.grandTotal),
      currency: raw.price.currency,
    },
    cabinClass: raw.travelerPricings[0]
      .fareDetailsBySegment[0].cabin,
    rawSupplierData: raw, // Keep original for booking step
  };
}

The rawSupplierData field is important. When the user selects a result and proceeds to booking, the booking service needs the original supplier payload, not our normalized version. Normalizing for display and keeping raw data for transactions is a pattern that saved us from dozens of edge case bugs.

NDC support: plan for it now

If your platform handles flights, you need to account for NDC (New Distribution Capability). Airlines like Lufthansa, American Airlines, and Singapore Airlines increasingly distribute their best fares through NDC channels rather than traditional GDS. Amadeus supports NDC through their Flight Offers Search API, but the response structure has subtle differences from GDS results. Our normalization layer handles both, but it required explicit branching logic.

The practical impact: if you only integrate with traditional GDS channels, you may miss 15 to 30% of available fares on NDC-forward airlines. Build your normalizer to handle both from the start.

Lessons learned

1. GDS integration is 25 to 35% of total development effort. We underestimated it on our first project. Authentication, rate limiting, error handling, data normalization, and edge cases (codeshare flights, multi-city itineraries, mixed-cabin fares) consume far more time than the booking flow itself. Our team at Adamo Software now plans GDS integration as a dedicated workstream, not a subtask.

2. Cache aggressively, but with clear invalidation rules. Stale pricing leads to booking failures. We learned to set TTLs based on the product type: 5 minutes for flights (prices change frequently), 30 minutes for hotels (rates are more stable), and 2 hours for activities (rarely change intraday).

3. Never trust a GDS response blindly. Validate prices, check that segments connect logically, and verify that the fare class is actually bookable before showing it to the user. We encountered phantom availability, where the GDS reports seats available but the airline rejects the booking, roughly 2 to 3% of the time on certain carriers.

4. Build supplier-agnostic from day one. Your search interface, result schema, and booking flow should not know or care which GDS provided the data. When we added a fourth supplier six months later, it took two days instead of two weeks because the architecture supported it.

5. Monitor your look-to-book ratio. GDS providers watch this metric. If you are making thousands of search calls but very few bookings, your pricing terms can change. Our ratio stabilized at around 1:25 (one booking per 25 searches) after implementing the caching layers.

Wrapping up

Building a real-time travel search engine is less about connecting to an API and more about engineering around its constraints. Rate limits, response format inconsistencies, authentication quirks, and NDC fragmentation are the real challenges. The caching architecture alone saved us from blowing our API budget and delivered sub-second response times to users. If you are building an AI-powered travel booking platform and planning GDS integration, invest your time in the data layer and caching strategy first. The API calls themselves are the easy part.

How we designed an AI Agent workflow with fallback chains and human-in-the-loop

Adamo Software — Mon, 23 Mar 2026 03:40:48 +0000

If you've shipped an AI agent to production, you already know the uncomfortable truth: the demo works great, but real users find every edge case your prompt didn't anticipate. We ran into this exact problem when building an internal document processing agent for a healthcare client. The agent worked fine 85% of the time. The other 15% ranged from "slightly wrong" to "confidently hallucinated a patient ID that doesn't exist."

This post walks through the fallback architecture we built to handle those failures gracefully, without turning every request into a human review bottleneck.

The problem with linear agent workflows

Our first version was straightforward: user uploads a document, the LLM extracts structured fields, validates against a schema, and writes to the database. A single chain, no branching logic.

The failure math killed us. If each step in a 5-step workflow has 90% reliability, your end-to-end success rate drops to about 59%. Add more steps, and it gets worse fast.

We needed a system where failures at any step could be caught, rerouted, and resolved without restarting the entire pipeline.

The fallback chain pattern

Instead of a single LLM call per step, we implemented a tiered fallback chain. The concept is simple: try the primary approach, and if confidence drops below a threshold, cascade to the next option.

Here's the core logic in Python:

from dataclasses import dataclass
from typing import Any, Optional
import logging

logger = logging.getLogger(__name__)

@dataclass
class AgentResult:
    output: Any
    confidence: float
    model_used: str
    fallback_triggered: bool = False

class FallbackChain:
    def __init__(self, confidence_threshold: float = 0.7):
        self.threshold = confidence_threshold
        self.chain = []

    def add_handler(self, name: str, handler, min_confidence: float = 0.0):
        self.chain.append({
            "name": name,
            "handler": handler,
            "min_confidence": min_confidence
        })
        return self

    async def execute(self, input_data: dict) -> AgentResult:
        for i, step in enumerate(self.chain):
            try:
                result = await step["handler"](input_data)

                if result.confidence >= self.threshold:
                    result.fallback_triggered = (i > 0)
                    logger.info(
                        f"Step '{step['name']}' succeeded "
                        f"(confidence: {result.confidence:.2f})"
                    )
                    return result

                logger.warning(
                    f"Step '{step['name']}' below threshold "
                    f"({result.confidence:.2f} < {self.threshold})"
                )

            except Exception as e:
                logger.error(f"Step '{step['name']}' failed: {e}")
                continue

        # All automated steps failed, escalate to human
        return AgentResult(
            output=None,
            confidence=0.0,
            model_used="human_escalation",
            fallback_triggered=True
        )

We typically set up three tiers:

Primary model (e.g., GPT-4o or Claude) with a specialized prompt. Fast, cost-effective for straightforward cases.
Enhanced model with additional context injection. We pull in RAG-retrieved examples of similar documents and few-shot them into the prompt.
Human escalation. The request lands in a review queue with full context: the original input, what each model attempted, and where confidence dropped.

# Setting up the chain for document extraction
extraction_chain = FallbackChain(confidence_threshold=0.75)

extraction_chain.add_handler(
    name="primary_extraction",
    handler=primary_llm_extract
)
extraction_chain.add_handler(
    name="enhanced_extraction_with_rag",
    handler=rag_enhanced_extract
)
extraction_chain.add_handler(
    name="human_review",
    handler=queue_for_human_review
)

Confidence scoring: the hard part

The fallback chain is useless without reliable confidence signals. LLM token probabilities alone are not enough. A model can be confidently wrong. Anthropic published a practical guide on evaluating agent outputs that covers calibration in more depth.

We use a composite confidence score built from three signals:

def compute_confidence(
    llm_output: dict,
    schema: dict,
    historical_outputs: list[dict]
) -> float:
    # 1. Schema compliance: does the output match expected types/formats?
    schema_score = validate_against_schema(llm_output, schema)

    # 2. Self-consistency: run the same input 3 times, 
    #    measure agreement across outputs
    consistency_score = measure_output_consistency(
        llm_output, historical_outputs
    )

    # 3. Field-level heuristics: known patterns for dates, IDs, codes
    heuristic_score = run_field_heuristics(llm_output)

    # Weighted combination
    return (
        0.3 * schema_score 
        + 0.4 * consistency_score 
        + 0.3 * heuristic_score
    )

Schema compliance catches obvious failures like missing required fields or wrong data types. Self-consistency catches the subtler ones. If you run the same extraction three times and get three different patient names, something is off.

The heuristic layer handles domain-specific validation. In healthcare, that means checking date formats, verifying that ICD codes match known patterns, and flagging values that fall outside clinical ranges.

Where human-in-the-loop actually fits

The biggest mistake we made early on was treating HITL as a binary switch: either the agent handles it or a human does. In practice, you need multiple levels of human involvement.

We settled on three escalation tiers:

Tier 1: Async review. The agent completed the task but confidence was borderline (0.6 to 0.75). A human reviewer sees the output alongside the original document and either approves or corrects it. This handles about 10% of requests and adds 2 to 4 hours of latency, which was acceptable for our use case.

Tier 2: Real-time intervention. Confidence dropped below 0.6, or the agent hit a known ambiguity pattern (e.g., handwritten notes, poor scan quality). The workflow pauses, and the request routes to an available specialist through a Slack notification. We used LangGraph's interrupt() pattern for this:

from langgraph.types import interrupt

def extraction_node(state: dict) -> dict:
    result = await extraction_chain.execute(state["document"])

    if result.model_used == "human_escalation":
        # Pause the workflow and wait for human input
        human_response = interrupt({
            "reason": "Low confidence extraction",
            "document_id": state["document_id"],
            "attempted_output": result.output,
            "confidence": result.confidence
        })
        return {"extracted_data": human_response}

    return {"extracted_data": result.output}

For a step-by-step walkthrough of interrupts and commands in LangGraph, this tutorial covers the basics well

Tier 3: Full manual processing. The document type is entirely outside the agent's training distribution. This happens maybe 2% of the time. The system logs the case as a training candidate for future model improvement.

Circuit breakers for cascading failures

One thing we learned the hard way: when an upstream model starts degrading (rate limits, API instability, model drift), it can poison every downstream step. A hallucinated field in step 1 becomes a corrupted database entry by step 4.

We added circuit breakers that monitor rolling error rates per step:

class CircuitBreaker:
    def __init__(self, failure_threshold: int = 5, reset_timeout: int = 60):
        self.failure_count = 0
        self.threshold = failure_threshold
        self.reset_timeout = reset_timeout
        self.state = "closed"  # closed = normal, open = blocking
        self.last_failure_time = None

    def record_failure(self):
        self.failure_count += 1
        self.last_failure_time = time.time()
        if self.failure_count >= self.threshold:
            self.state = "open"
            logger.critical("Circuit breaker OPEN. Routing to fallback.")

    def can_execute(self) -> bool:
        if self.state == "closed":
            return True
        # Check if enough time has passed to retry
        if time.time() - self.last_failure_time > self.reset_timeout:
            self.state = "half-open"
            return True
        return False

When the circuit opens, all requests for that step skip directly to the next fallback tier. This prevents a degraded model from wasting tokens and time on requests it's going to fail anyway.

What we measured

After running this architecture for three months on the healthcare document processing pipeline:

End-to-end accuracy went from 85% to 97.3%
Average latency increased by 400ms for the primary path (acceptable)
Human review volume dropped from ~30% of all documents to ~12%, because the enhanced RAG fallback caught most borderline cases
Zero hallucinated patient IDs made it to the database (previously ~2 per week)

The biggest win was not the accuracy improvement itself. It was the fact that we could now quantify exactly where the system was failing and allocate human attention to the cases that actually needed it.

Key takeaways

Design for failure from day one. If your agent workflow has no fallback path, you're building a demo, not a production system.
Confidence scoring needs multiple signals. Token probabilities are not enough. Combine schema validation, self-consistency checks, and domain heuristics.
HITL is a spectrum, not a switch. Different confidence levels should trigger different levels of human involvement. Not every edge case needs real-time intervention.
Monitor the monitors. Circuit breakers and rolling error rates prevent cascading failures from eating your entire pipeline.

Wrapping up

Building reliable AI agent workflows is less about picking the right model and more about designing the right failure modes. The fallback chain pattern gave us a structured way to degrade gracefully, and the tiered HITL approach kept humans involved where they add value without turning them into full-time babysitters for the AI.

If you want to dive deeper into interrupt mechanics, the LangGraph team wrote a solid overview of the pattern.

If you're building something similar, start with the confidence scoring. Everything else follows from having a reliable signal for "how much should I trust this output."

I'm a software engineer at Adamo Software, where we build custom AI and healthcare platforms for enterprise clients.