Integrating Amazon S3 Vectors as a Persistent Memory Backend for NVIDIA NeMo Agent Toolkit

Amazon S3 Vectors can now be used as a custom persistent memory backend for the NVIDIA NeMo Agent Toolkit (NAT). This change lets teams replace the built‑in providers with a vector store that offers elastic scale, strong write consistency, and pay‑as‑you‑go pricing, all while staying within the same Kubernetes deployment on Amazon EKS.

Why S3 Vectors fits the NAT memory requirements

The NAT memory subsystem expects a backend that can store MemoryItem objects and retrieve them by semantic similarity and metadata filters. S3 Vectors provides:

• Vector similarity search with configurable distance metrics such as cosine and Euclidean.
• Rich, filterable metadata (strings, numbers, Booleans, lists) that satisfies NAT’s scoped‑query needs.
• Strong write consistency, ensuring that a newly added memory item is visible to all agents immediately.
• Capacity up to two billion vectors per index, removing the need for capacity planning.
• Cost model based solely on storage, writes, and queries, eliminating idle compute charges.
• IAM‑based access control at the bucket and index level, allowing per‑tenant isolation when required.

Implementing a custom S3 Vectors provider

NAT’s plugin model requires a class that implements the MemoryEditor abstract interface. The three required methods are add_items(), search(), and remove_items(). A typical implementation follows these steps:

1. Create a subclass of MemoryBaseConfig that adds any provider‑specific settings and sets the _type field to a unique identifier (e.g., s3_vectors).
2. In add_items(), serialize each MemoryItem into the JSON format expected by S3 Vectors and invoke the S3 Vectors PutItem API.
3. In search(), translate the incoming query vector and metadata filters into the S3 Vectors Query API call, then map the response back to NAT’s MemoryItem objects.
4. In remove_items(), call the S3 Vectors DeleteItem API for the specified identifiers.

Because the provider is discovered via the _type field in the NAT YAML configuration, swapping the backend only requires updating the config file and redeploying the service.

Deploying NAT with the new provider on Amazon EKS

The deployment pattern remains unchanged: NAT runs as a set of containers on an Amazon EKS cluster, managed by standard Kubernetes primitives. The steps to bring the S3 Vectors backend online are:

• Provision an S3 bucket and enable the Vectors feature, creating an index that matches the expected dimensionality of the agent embeddings.
• Assign IAM policies that allow the EKS service role to perform PutItem, Query, and DeleteItem on the bucket and index.
• Package the custom provider code into a container image and reference it in the NAT deployment manifest.
• Configure the NAT auto_memory_agent workflow to use the new provider via the updated YAML config.

Once the pods start, agents automatically capture conversation snippets, store them in S3 Vectors, and retrieve relevant memories on subsequent turns without any code changes in the agent logic.

Related CloudNinjas coverage: AWS.

What This Means For Practitioners

Adopting S3 Vectors as the memory layer removes the need to manage separate vector databases or in‑memory caches, simplifying operations and reducing cost. Engineers gain immediate consistency across all agents, which is critical for coordinated multi‑agent workflows. Security teams can rely on existing IAM policies to enforce tenant isolation, but they should still audit bucket permissions and monitor S3 access logs. The primary evaluation points are latency of S3 Vectors queries at scale and the operational overhead of managing index lifecycle within the S3 bucket.