NVIDIA releases Nemotron 3.5 Lightning and open-source Switchyard for agent routing

NVIDIA has expanded its Nemotron 3 model family with Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model aimed at fast, specialized tasks inside long-running AI agent workflows. The company also released NeMo Switchyard, an open-source routing library designed to send each step of an agent task to the most suitable model across a developer's chosen mix of open, proprietary and NVIDIA-backed systems.

The announcement matters because it targets two practical limits in agent deployments: latency and cost. NVIDIA says Nemotron 3.5 Lightning is built for high-volume work such as code review, tool use, alert monitoring and domain-specific question answering, while larger reasoning models can still handle planning or orchestration. In NVIDIA's framing, agent systems increasingly look less like one all-purpose chatbot and more like a coordinated set of models with different jobs.

What NVIDIA Claims

• Nemotron 3.5 Lightning is a customizable open model with 30 billion parameters using a mixture-of-experts architecture.
• The model can run on local systems including RTX PCs, DGX Spark, DGX Station and Jetson devices, as well as workstations, data centers and cloud environments.
• NVIDIA says the model delivers up to four times faster output speed and 30% faster agentic task completion compared with other models in its class.
• NeMo Switchyard is available as an open-source project and can route requests using configurable strategies such as quality, latency and cost priorities.

For enterprises, the main pitch is operational control. Keeping smaller task-specific models closer to local data can reduce cloud inference dependency for some workflows, while model routing may help teams avoid sending every request to an expensive frontier model. NVIDIA cites internal benchmarks showing Switchyard can maintain frontier-level accuracy while cutting task completion cost to nearly one-third of using Opus 4.8 alone, though that remains a company benchmark rather than an independent result.

The public Switchyard repository describes the project as a Rust proxy and library for LLM traffic. It translates between OpenAI and Anthropic-style APIs, records operational metrics and supports multiple routing approaches. The repository also labels the software pre-alpha and experimental, so production teams should treat the release as an early developer tool rather than a mature platform component.

The broader signal is that AI infrastructure vendors are moving beyond headline model size and into deployment mechanics. If agents are expected to run for hours, touch private data and call tools repeatedly, routing, local execution and model specialization become part of the product, not optional plumbing.

Sources

• blogs.nvidia.com
• github.com

Cover photo by Michal Hajtas on Pexels, used under the Pexels License.