Amazon SageMaker AI now accepts an ordered list of up to five instance types when you create a training or processing job. The service evaluates the list at launch time, picks the first type with available capacity, and starts the job without the need for manual retries or separate scripts.
Feature Overview
The new instance preference list is a simple extension to the existing job definition. Instead of a single InstanceType field, you provide an array of acceptable types in priority order. SageMaker checks each entry sequentially, first attempting any reserved capacity (such as Flexible Training Plans) before falling back to on‑demand capacity for the next type in the list.
Operational Impact
Prior to this change, teams often wrote custom retry loops that polled job status, cancelled stalled submissions, and resubmitted with alternative instance types. Those scripts added operational overhead and were fragile when capacity patterns shifted. With preference lists the platform handles the fallback automatically, which reduces:
- Waiting time during peak demand periods.
- Complexity of monitoring and retry logic in CI/CD pipelines.
- Potential for human error in manual resubmissions.
The result is faster job start times and higher overall capacity utilization, letting engineers focus on model development rather than capacity gymnastics.
Architectural Considerations
Adopting instance preference lists does not require changes to the underlying training or processing code, but it does affect how you design job submission workflows:
- Ensure that all listed instance types are compatible with the container image, framework version, and any hardware‑specific optimizations you rely on.
- When using Flexible Training Plans, the list should place the reserved instance type first so the reservation is evaluated before on‑demand fallbacks.
- The API now accepts an array; update any automation that builds the request payload to include the new field.
- Capacity planning can be simplified because the platform will automatically balance demand across the preferred types, but you may still want to monitor which types are selected most often to inform future instance selections.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Replace custom retry scripts with a single job definition that includes an ordered list of up to five acceptable instance types. Verify compatibility across the list, prioritize reserved capacity where applicable, and update your CI/CD templates to use the new parameter. Monitor the chosen instance types to confirm the fallback behavior aligns with cost and performance expectations, and adjust the list order as your workload patterns evolve.
