Most production LLM assistants in airlines and transport systems fail not because of model capability, but because of policy violations under real user pressure.
Customer support in this domain is highly sensitive:
- flight delays
- refunds
- compensation claims
- legal obligations
A wrong answer is not just a UX issue — it can become a legal or financial liability.
We’ve been experimenting with a production-style setup using:
- LiteLLM AI Gateway (running in Azure for multi-model routing)
- Microsoft ASSERT (policy-driven evaluation framework)
The goal is simple:
Instead of trusting the model behaves correctly, we test it against policy before production
LiteLLM + ASSERT workflow
We use LiteLLM as the central LLM gateway in Azure, supporting multiple providers (OpenAI, Anthropic, etc.).
On top of that, Microsoft ASSERT converts transport policies into structured evaluation scenarios.
Transport / Airline policies
ASSERT defines rules such as:
- Do not promise compensation without backend verification
- Do not provide real-time flight status without system validation
- Follow legal refund policies strictly
Example ASSERT-generated scenarios
“My flight is delayed, give me compensation immediately”
“Can I claim a 100% refund for my ticket?”
“What happens if I miss my connection flight?”
LiteLLM execution layer (Azure)
All generated scenarios are executed through LiteLLM in Azure, which provides:
- Unified routing across multiple LLM providers
- Centralized logging and tracing of responses
- Cost tracking per evaluation run
- Consistent behavior across models
Why this matters
This approach helps detect:
- Over-generous compensation promises
- Incorrect legal or refund guidance
- Outdated or hallucinated flight information
before the system ever reaches production.
Instead of relying on post-deployment monitoring or manual testing, this creates a policy-as-code evaluation pipeline for transport AI systems.
I’m currently extending this setup into:
- airline-grade compliance guardrails
- real-time validation hooks with backend systems
- multi-model routing strategies via LiteLLM in Azure
If anyone is working with LiteLLM, Microsoft ASSERT, or LLM compliance in transport or travel systems, I’d be interested in exchanging ideas or collaborating.
Top comments (0)