Amazon SageMaker AI has added native support for Multi‑Turn Reinforcement Learning (MTRL), allowing a policy model to be trained on whole‑episode outcomes instead of isolated prompts. Practitioners can now host an agent on AgentCore (or on self‑managed compute such as EKS, EC2, or Fargate via an AWS Lambda forwarder), let the agent interact with a task environment, and receive a single reward at episode completion. The shift from single‑turn fine‑tuning to episode‑level reinforcement opens a path for building domain‑specific agents that require sequential decision making.
Solution Overview
The MTRL flow consists of two logical halves. SageMaker AI owns the training job and the policy model, while AgentCore hosts the agent code, the task environment, and the reward function. For each rollout, SageMaker invokes the AgentCore runtime with one secret word (the Wordle target in the example). The agent calls the policy model through SageMaker’s OpenAI‑compatible endpoint, makes up to six guess_word calls, receives per‑letter feedback, and finally returns a scalar reward. The reward is outcome‑dominant: a solve yields 1.05–1.55 points based on speed, a loss yields –0.5, and an invalid guess yields –1.5, with a small shaping term for intermediate strategy.
Training Pipeline Details
The pipeline uses Group Relative Policy Optimization (GRPO) to compare reward spreads across multiple rollouts of the same secret word. Updates are applied to a Low‑Rank Adaptation (LoRA) adapter attached to the base model. In the published run, the GPT-OSS-20B model improved its Wordle solve rate from 61 % to 84 % at default reasoning effort, and from 15 % to 69 % at a low‑effort setting, for a token cost of roughly $339. The dataset supplied to the service contains only the secret word; all environment logic and scoring reside in a single Python file deployed to AgentCore. Supported base models at the time of writing include Amazon Nova 2 Lite and GPT-OSS-20B, available in US East (N. Virginia) and US West (Oregon) regions.
Operational and Security Considerations
Because the agent and reward function run together in a single deployment unit, the attack surface is limited to the code package you upload. Practitioners should treat the Python file as a trusted artifact and apply standard code‑review and supply‑chain checks before deployment. The forwarder Lambda acts only as a network bridge; its permissions should be scoped to invoke the specific AgentCore runtime and to read/write the SageMaker training job resources. Data residency stays within the chosen AWS region, and the training tokens are billed to the account that owns the SageMaker job, keeping cost attribution clear. Monitoring can be hooked into MLflow as demonstrated, providing visibility into reward trends and adapter weight changes.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Multi‑Turn RL on SageMaker AI gives engineers a serverless path to train low‑cost, task‑specific agents without exposing proprietary data to base‑model training. The architecture isolates the environment and reward logic, simplifies credential management to a single Lambda forwarder, and leverages familiar MLOps tooling for observability. Teams should evaluate whether their workflow requires sequential decision making, prototype the agent in AgentCore, and use the provided reward design patterns as a starting point. Ongoing evaluation of solve rates, token spend, and reward shaping will be essential to keep the model both effective and cost‑efficient.
