As enterprises adopt AI for software development, one challenge keeps surfacing:
How do you deliver frontier-level coding performance while keeping code, intellectual property, and development context inside enterprise-controlled environments?
That question became especially important as IBM expanded IBM Bob into self-hosted deployments. While cloud-based AI services can provide powerful capabilities, many organizations operate under strict security, compliance, sovereignty, and operational requirements that make sending sensitive code to external environments impractical.
To address this, IBM needed models capable of delivering enterprise-grade agentic development while running efficiently on customer-managed infrastructure.
After extensive evaluation, IBM selected NVIDIA Nemotron and Poolside Laguna as key supported models for IBM Bob self-hosted deployments.
The Challenge: Bringing Frontier AI On-Premises
Self-hosted AI creates a very different set of requirements compared to public SaaS deployments.
Organizations operating in regulated industries often need:
- Source code to remain inside approved environments
- Development artifacts to stay within enterprise boundaries
- Air-gapped deployment options
- Flexible infrastructure control
- Predictable performance and latency
- Governance over model usage
At the same time, developers still expect modern AI capabilities.
The solution cannot simply be secure.
It also needs to be effective.
That balance became the driving factor behind the model evaluation process.
How IBM Evaluated Candidate Models
Rather than choosing models based solely on benchmark scores, the IBM Bob team evaluated both open-weight and closed-weight models across several dimensions that matter in real enterprise environments.
1. Hardware Footprint
Many frontier models require enormous computational resources.
While those models may produce excellent results, deploying them on-premises can become prohibitively expensive.
IBM evaluated models across different size profiles to determine which models provided the best balance between intelligence and infrastructure requirements.
The goal was practical deployment, not simply maximal model size.
2. Coding Accuracy
A coding model ultimately needs to help solve software engineering problems.
IBM benchmarked candidate models across real-world development workflows, including:
- Simple development tasks
- Medium-complexity engineering work
- Multi-step software modernization scenarios
- Agent-assisted workflows
The emphasis was on actual task completion rather than isolated benchmark scores.
3. Latency
Developer productivity depends heavily on feedback loops.
When an agent performs analysis, tool calls, file operations, and execution workflows, slow responses create friction.
IBM therefore measured:
- Time-to-first-token
- Overall response latency
- Generation throughput
- Multi-step workflow responsiveness
4. Openness
A self-hosted deployment requires more than raw capability.
Organizations need models they can govern, deploy, and operate within their own environments.
This makes open-weight models especially attractive for customer-managed deployments, including highly controlled and air-gapped environments.
Looking Beyond Raw Model Benchmarks
One particularly interesting aspect of the evaluation process was that IBM did not evaluate models in isolation.
The Bob team evaluated models together with the surrounding agent harness.
This matters because agentic development extends beyond model reasoning.
Real-world workflows involve:
- Tool calling
- Terminal interactions
- Planning systems
- Skills
- Modes
- Agent orchestration
A model that performs well in benchmarks may not necessarily perform well inside a complete software engineering workflow.
The evaluation therefore focused on end-to-end performance.
Co-Engineering with NVIDIA
During early testing, NVIDIA Nemotron models demonstrated strong reasoning capabilities.
However, IBM identified opportunities to improve their effectiveness within agent-based software development workflows.
The IBM Bob engineering team collaborated closely with NVIDIA to refine the behavior of the models within Bob's environment.
Optimization efforts included:
- Prompt tuning
- Sampling adjustments
- Reasoning configuration improvements
- Agent harness modifications
The process wasn't simply about adapting Bob to the model.
It was about co-engineering the complete experience.
Feedback from IBM engineers and internal Bob users was continuously incorporated into the evaluation and optimization process.
What IBM Found
NVIDIA Nemotron 3 Ultra
NVIDIA Nemotron 3 Ultra stood out in several areas.
IBM highlighted:
- Reliable and predictable output behavior
- Strong adherence to tool schemas
- Reduced tendency to drift off-task
- Clean implementations without unnecessary complexity
- Low-latency execution characteristics
According to IBM's internal benchmarks, Nemotron delivered significantly lower latency on NVIDIA Blackwell infrastructure compared with leading SaaS-based model deployments accessed over the network.
For enterprise development teams, lower latency can translate directly into faster feedback cycles and smoother agent interactions.
Poolside Laguna S 2.1
Poolside Laguna impressed for different reasons.
IBM highlighted:
- Strong reasoning relative to model size
- Fast response generation
- Clear explanations of reasoning processes
- Excellent performance-to-footprint ratio
This makes Laguna particularly attractive in situations where infrastructure resources are more constrained but high-quality coding assistance remains necessary.
What This Means for Enterprise Customers
For organizations considering self-hosted AI-assisted software development, the outcome of this work is significant.
With IBM Bob self-hosted, organizations can deploy supported models directly on their own infrastructure and connect them through Bob's model gateway.
The practical benefits include:
- Code stays within enterprise-controlled environments
- Development context remains local
- Intellectual property remains protected
- Air-gapped deployments become possible
- Enterprises retain infrastructure control
- Development teams gain access to agentic workflows
Perhaps more importantly, enterprises no longer need to choose between strong AI capabilities and deployment flexibility.
The combination of IBM Bob with models such as NVIDIA Nemotron and Poolside Laguna aims to make frontier-grade software engineering assistance available inside regulated and security-sensitive environments.
Looking Ahead
IBM also outlined future plans to expand model flexibility further.
One particularly interesting direction is multi-model routing.
As enterprises adopt multiple models for different purposes, routing workloads to the most appropriate model could help balance:
- Cost
- Performance
- Accuracy
- Infrastructure utilization
This aligns with a broader industry trend where organizations increasingly use multiple models rather than relying on a single AI provider.
Final Thoughts
The most interesting aspect of this announcement is that it shifts the conversation away from model rankings and toward deployment reality.
Many organizations already know AI can improve software development.
The challenge is bringing those capabilities into environments where their most important applications live.
IBM's decision to support NVIDIA Nemotron and Poolside Laguna reflects a practical enterprise requirement:
Deliver strong agentic development capabilities while preserving control over infrastructure, code, data, and operational environments.
For regulated industries, air-gapped environments, and enterprises managing sensitive intellectual property, that combination may be just as important as model intelligence itself.
Learn More
📖 Read the full announcement:
👉 Why IBM Chose NVIDIA Nemotron for IBM Bob Self-Hosted
⤵️ Try IBM Bob for free: 👉 Start Your IBM Bob Trial
Top comments (0)