Live
Local Artifacts Support Lets Workers Test Repo Workflows Before DeployCo‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI ClusterScaling AI Agents on Kubernetes: New CNCF Projects and Operational ImplicationsGitHub Copilot model deprecation forces updates to AI‑assisted development pipelinesGitHub Copilot expands Enterprise Server access and adds new AI models – practical impact for engineersArchitecting Agentic Customer Engagement with Amazon Bedrock AgentCoreBuilding Multi‑Tenant Memory‑Aware Agents with Bedrock Agentic Retrieve and Managed Knowledge BaseBulk‑Start AI Scan for Up to 20 GitHub Repos: What Engineers Need to KnowLocal Artifacts Support Lets Workers Test Repo Workflows Before DeployCo‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI ClusterScaling AI Agents on Kubernetes: New CNCF Projects and Operational ImplicationsGitHub Copilot model deprecation forces updates to AI‑assisted development pipelinesGitHub Copilot expands Enterprise Server access and adds new AI models – practical impact for engineersArchitecting Agentic Customer Engagement with Amazon Bedrock AgentCoreBuilding Multi‑Tenant Memory‑Aware Agents with Bedrock Agentic Retrieve and Managed Knowledge BaseBulk‑Start AI Scan for Up to 20 GitHub Repos: What Engineers Need to Know
AWS

Integrating Amazon S3 Vectors as a Persistent Memory Backend for NVIDIA NeMo Agent Toolkit

AI SummaryPowered by AI

NVIDIA NeMo Agent Toolkit now supports Amazon S3 Vectors as a custom persistent memory backend. This gives multi‑agent deployments elastic vector storage, strong consistency, and cost‑effective scaling on Amazon EKS.

Amazon S3 Vectors can now be used as a custom persistent memory backend for the NVIDIA NeMo Agent Toolkit (NAT). This change lets teams replace the built‑in providers with a vector store that offers elastic scale, strong write consistency, and pay‑as‑you‑go pricing, all while staying within the same Kubernetes deployment on Amazon EKS.

Why S3 Vectors fits the NAT memory requirements

The NAT memory subsystem expects a backend that can store MemoryItem objects and retrieve them by semantic similarity and metadata filters. S3 Vectors provides:

  • Vector similarity search with configurable distance metrics such as cosine and Euclidean.
  • Rich, filterable metadata (strings, numbers, Booleans, lists) that satisfies NAT’s scoped‑query needs.
  • Strong write consistency, ensuring that a newly added memory item is visible to all agents immediately.
  • Capacity up to two billion vectors per index, removing the need for capacity planning.
  • Cost model based solely on storage, writes, and queries, eliminating idle compute charges.
  • IAM‑based access control at the bucket and index level, allowing per‑tenant isolation when required.

Implementing a custom S3 Vectors provider

NAT’s plugin model requires a class that implements the MemoryEditor abstract interface. The three required methods are add_items(), search(), and remove_items(). A typical implementation follows these steps:

  1. Create a subclass of MemoryBaseConfig that adds any provider‑specific settings and sets the _type field to a unique identifier (e.g., s3_vectors).
  2. In add_items(), serialize each MemoryItem into the JSON format expected by S3 Vectors and invoke the S3 Vectors PutItem API.
  3. In search(), translate the incoming query vector and metadata filters into the S3 Vectors Query API call, then map the response back to NAT’s MemoryItem objects.
  4. In remove_items(), call the S3 Vectors DeleteItem API for the specified identifiers.

Because the provider is discovered via the _type field in the NAT YAML configuration, swapping the backend only requires updating the config file and redeploying the service.

Deploying NAT with the new provider on Amazon EKS

The deployment pattern remains unchanged: NAT runs as a set of containers on an Amazon EKS cluster, managed by standard Kubernetes primitives. The steps to bring the S3 Vectors backend online are:

  • Provision an S3 bucket and enable the Vectors feature, creating an index that matches the expected dimensionality of the agent embeddings.
  • Assign IAM policies that allow the EKS service role to perform PutItem, Query, and DeleteItem on the bucket and index.
  • Package the custom provider code into a container image and reference it in the NAT deployment manifest.
  • Configure the NAT auto_memory_agent workflow to use the new provider via the updated YAML config.

Once the pods start, agents automatically capture conversation snippets, store them in S3 Vectors, and retrieve relevant memories on subsequent turns without any code changes in the agent logic.

Related CloudNinjas coverage: AWS.

What This Means For Practitioners

Adopting S3 Vectors as the memory layer removes the need to manage separate vector databases or in‑memory caches, simplifying operations and reducing cost. Engineers gain immediate consistency across all agents, which is critical for coordinated multi‑agent workflows. Security teams can rely on existing IAM policies to enforce tenant isolation, but they should still audit bucket permissions and monitor S3 access logs. The primary evaluation points are latency of S3 Vectors queries at scale and the operational overhead of managing index lifecycle within the S3 bucket.

Originally published atAWS Machine Learning Blog