Amazon SageMaker JumpStart has been updated to host NVIDIA Nemotron 3.5 Super VL, a 120‑billion‑parameter vision‑language model that can ingest images, video, and text and emit textual responses. The service now exposes two checkpoints – a quantized NVFP4 version (4‑bit weights/activations) and a BF16 reference – and provides pre‑configured instance types for inference. For engineers, the change means a managed entry point for large‑scale multimodal workloads without having to provision the model infrastructure manually.
Model variants and resource footprint
Both variants share the same architecture: a hybrid Mixture‑of‑Experts (MoE) language backbone with 12 B active parameters per token and a C‑RADIO vision encoder. The NVFP4 checkpoint compresses the model to roughly 75 GB and runs on ml.g7.24xlarge (default) or ml.g7.48xlarge. The BF16 checkpoint is about 249 GB and requires ml.p5en.48xlarge. Context length supports up to one million tokens, though JumpStart serving configurations currently limit requests to 520–15,600 tokens. Both variants accept image, video, and text inputs and return text, with optional tool‑calling and JSON schema output.
Operational considerations
Deploying Nemotron 3.5 Super VL through JumpStart eliminates the need to manage GPU drivers, model sharding, or custom Docker images. However, the large instance sizes imply higher cost per inference hour, and the NVFP4 variant’s speculative decoding (MTP head) is not enabled in the default serving configuration, so latency expectations should be based on standard decoding paths. The service does not currently support LoRA adapters, so fine‑tuning or domain adaptation must be performed offline on the BF16 checkpoint before re‑uploading custom weights.
- Instance selection: Choose NVFP4 for latency‑critical, high‑throughput scenarios; choose BF16 when you need the full‑precision weights for downstream fine‑tuning.
- Payload limits: Each request can contain a single image and a single video; video content must be base64‑encoded and stay within the endpoint’s payload size limits.
- Reasoning toggle: The model enables reasoning by default; it can be disabled via the
enable_thinkingflag in the request payload.
Security and licensing implications
The model is distributed under the OpenMDW License Agreement version 1.1. Teams must verify that their use cases comply with the license terms, especially when integrating the model into commercial SaaS offerings. Because the service runs on managed AWS infrastructure, IAM policies governing SageMaker endpoint access remain the primary control surface; no additional authentication mechanisms are introduced by the model itself.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Engineers can now prototype multimodal agents—document intelligence, GUI understanding, or short‑video Q&A—directly from SageMaker without building a custom serving stack. The key actions are to evaluate the cost‑performance trade‑off between NVFP4 and BF16, provision the appropriate instance type, and incorporate the enable_thinking flag if configurable reasoning is required. For teams that need model customization, the BF16 checkpoint provides a starting point for fine‑tuning, but the resulting weights must be hosted outside JumpStart, as the managed endpoint does not load LoRA adapters. Finally, ensure that endpoint IAM policies and the OpenMDW license are reviewed before exposing the model to production workloads.
