Nvidia just expanded its Nemotron model family with two releases that could reshape how multi-agent AI systems are built: Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model designed for long-running agentic workloads, and NeMo Switchyard, an open-source routing library for intelligently directing requests across different models.
Nemotron 3.5 Lightning: Built for Agents, Not Chatbots
The shift from chatbots to autonomous agents is changing what we need from AI models. Chatbots handle one-off interactions — a question, an answer, done. Agents run continuously, making dozens of tool calls, maintaining context over long sessions, and executing multi-step workflows.
Nemotron 3.5 Lightning is specifically optimized for this use case. As a 30B-parameter mixture-of-experts (MoE) model, it activates only a subset of its parameters per token, giving it the efficiency of a much smaller model while maintaining the capability of a larger one. Key characteristics:
- High efficiency for sustained workloads — lower per-token cost makes long agent sessions economically viable
- 30B total parameters with sparse activation — the MoE architecture means only relevant experts fire for each token
- Open and customizable — enterprises can fine-tune and deploy it on their own infrastructure
- Edge-to-cloud deployment — designed to run across PCs, workstations, data centers, and cloud
This matters because agent economics are brutal. A chatbot serving one response per user session is cheap. An agent making 50 tool calls, each requiring model inference, multiplies costs dramatically. A model that's optimized for efficiency in long-running contexts could be the difference between an agent product that's profitable and one that isn't.
NeMo Switchyard: Smart Model Routing
The second release, NeMo Switchyard, might actually be the more interesting of the two. It's an open-source library for intelligent model routing that integrates with popular agent frameworks.
The problem Switchyard solves: most agent systems use a single model for everything, which is wasteful. A simple tool call doesn't need a frontier model, and a complex reasoning step shouldn't use a lightweight model. Switchyard acts as a traffic controller, directing each request to the most appropriate model based on the task's requirements.
Key features:
- Pluggable routing logic — build custom routers based on your specific needs
- Multi-model support — route across open models, proprietary models, and Nvidia models simultaneously
- No application rewrites — integrates with existing agent frameworks without requiring code changes
- Open source — fully customizable and auditable
This is significant because it enables a cost-optimization pattern that every agent system should be thinking about: use the cheapest model that can handle each step, and only escalate to expensive models when needed.
The Bigger Picture: AI Infrastructure Is Specializing
These releases are part of a broader trend: AI infrastructure is specializing for specific workload types. The era of "one model for everything" is ending. Instead, we're seeing:
- Agent-optimized models like Nemotron 3.5 Lightning, designed for sustained multi-step operations
- Routing layers like Switchyard, that orchestrate across model tiers
- Edge deployment targets — running models on phones, laptops, and IoT devices
- Domain-specific fine-tunes — models optimized for particular industries or tasks
For developers building agent systems, this specialization is good news. It means you're no longer stuck paying frontier-model prices for every inference call. With smart routing and efficient specialized models, you can build agent products with sustainable unit economics.
Should You Use These?
If you're building multi-agent systems, NeMo Switchyard is worth evaluating immediately — the routing pattern it enables is something most agent systems should implement regardless of the specific library. The ROI of routing simple tasks to cheaper models can be 5-10x cost reduction with minimal quality impact.
Nemotron 3.5 Lightning is worth watching if you're doing long-running agent workloads. The MoE architecture is well-suited for sustained inference, and the open nature means you can fine-tune and deploy it where it makes sense for your infrastructure.
Both are available now, with documentation on Nvidia's developer portal.
Based on Nvidia's announcement by Kari Briski, August 2026. Learn more at Nvidia's blog.
Top comments (0)