Nvidia’s latest Nemotron 3.5 family adds two practical shifts: a 120‑billion‑parameter Super VL model that adds image and video input, and a 74‑billion‑parameter Bolt model that fits into 24 GB of GPU memory through two‑bit quantization. Both are positioned for on‑device agentic tasks, meaning engineers can now run sophisticated reasoning, tool calling, and web‑browsing workloads on laptops or edge GPUs rather than large‑scale clusters.
What Changed
Super VL expands the earlier Nemotron 3 Super model by keeping the 120 B total parameter count but activating only 12 B during inference and adding a vision encoder for image and video inputs. It retains a hybrid architecture that mixes Mamba‑2, attention, and mixture‑of‑experts (MoE) layers. Nvidia also introduced LiLiCorr speculative decoding, where a lightweight helper drafts multiple tokens ahead, yielding up to 2.3× faster generation on DGX Spark.
The Bolt 74 B‑A9B variant compresses the model to a 24 GB footprint using two‑bit quantization and activates 9 B parameters at a time. It inherits the hybrid MoE design and is optimized for RTX Spark and DGX Spark systems, with support for inference runtimes such as llama.cpp, Ollama, LM Studio, and vLLM. A smaller 4 GB Bolt E4B model is slated for release later in the month.
Why It Matters for Engineers
Local deployment reduces latency, data‑transfer costs, and reliance on cloud‑based inference APIs, which is attractive for privacy‑sensitive or bandwidth‑constrained environments. The performance numbers show Bolt scoring 51.0 on Terminal‑Bench 2.1, surpassing comparable open models (e.g., Qwen 3.6‑35B‑A3B at 44.4). Super VL’s fourfold speedup on 10 k SWE‑bench Pro tasks across four B200 GPUs demonstrates that the hybrid architecture can deliver higher throughput without proportionally increasing hardware size.
Architectural and Operational Implications
The hybrid MoE design means only a subset of parameters is active per token, which aligns well with quantization strategies and can lower memory bandwidth pressure. Two‑bit quantization, however, introduces a trade‑off between model size and numerical precision; engineers should validate accuracy for their specific workloads, especially where fine‑grained reasoning or code generation is critical.
Speculative decoding (LiLiCorr) adds a helper model to the inference pipeline. Deployments must provision additional compute for the helper, but the net token‑generation latency improves. Integration with existing stacks is straightforward because Nvidia ships the models for common runtimes, but practitioners need to ensure their inference servers (e.g., vLLM) are configured to handle mixed‑precision and MoE routing.
Security and Compliance Considerations
Both models are distributed on Hugging Face under the OpenMDW 1.1 license, which explicitly permits commercial use and modification. Teams must track license compliance, especially when integrating the models into proprietary products or when redistributing derived artifacts. No explicit security controls or vulnerabilities are mentioned, so standard model‑deployment hardening practices (e.g., sandboxed inference, input validation) remain applicable.
Related CloudNinjas coverage: AI engineering.
What This Means For Practitioners
- Evaluate Bolt for laptop‑scale agentic workloads where latency and data residency matter.
- Benchmark the two‑bit quantized model against existing 4‑bit or 8‑bit alternatives to quantify the accuracy‑performance trade‑off for your use case.
- Leverage the supported runtimes (
llama.cpp, Ollama, LM Studio, vLLM) to integrate the model into CI/CD pipelines for continuous evaluation. - Monitor the upcoming 4 GB Bolt E4B release for ultra‑lightweight edge scenarios.
- Maintain license tracking for OpenMDW 1.1 to ensure compliance in commercial deployments.
