Amazon Bedrock introduced a prompt‑caching feature that lets you persist the pre‑fill state of a prompt and reuse it on later calls. For AI agents that repeatedly send the same tool definitions, developer instructions, and accumulated conversation history, this change eliminates the need to re‑process identical tokens on every turn, directly lowering both cost and response time.
How Prompt Caching Works in Bedrock
When an LLM receives a prompt, it first performs a prefill step that converts each input token into key‑value (KV) vectors. These vectors represent the model’s working state and are normally recomputed for every request. Prompt caching captures this KV state after the static prefix—typically the tools list, system instructions, and any shared content—has been processed. On subsequent requests that start with the same token sequence, Bedrock can load the saved KV state instead of recomputing it, and then only processes the new portion of the payload.
Bedrock offers two modes: implicit caching, where the service automatically determines the reusable prefix, and explicit caching, where you mark the endpoint with the prompt_cache_breakpoint parameter. Combining both gives you a reliable baseline cache while allowing the service to manage growing conversation tails.
Applying Caching to Agent Workflows
In a typical agent loop, each turn includes:
- Tool list – static function definitions that rarely change.
- Instructions – developer‑provided rules, persona, and policy text.
- History – the accumulated dialogue, tool calls, and results.
- New turn – the latest user message or tool output.
Without caching, the entire block (tools, instructions, full history, and new turn) is sent on every request, causing token charges to grow linearly with conversation length. By placing a prompt_cache_breakpoint after the static tools and instructions—once the combined token count exceeds the 1,024‑token threshold required for caching—you ensure that only the new turn incurs full pricing. Implicit caching then picks up the expanding history, reusing the cached prefix automatically.
When a new tool is added or a sub‑agent is spawned, you can reuse the previously cached prefix across sessions. The new session starts by loading the shared static cache, then appends its own unique history, preserving cost savings while keeping each conversation isolated.
Operational and Security Considerations
From an operations perspective, prompt caching reduces the average request payload size, which can improve throughput on high‑concurrency workloads. Monitoring should focus on cache hit rates and the size of the cached prefix to verify that the breakpoint is positioned effectively.
Security implications are limited to the data stored in the cache. Because the cached KV state contains the exact text of the static prompt, any sensitive information embedded in tool definitions or policies would be persisted in Bedrock’s cache layer. Practitioners should avoid placing secrets in the cached portion and instead inject them at runtime after the breakpoint, ensuring they are never written to the reusable cache.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Implement explicit prompt_cache_breakpoint markers after static tool and instruction blocks, and let Bedrock handle implicit caching for the evolving conversation. Verify that the cached prefix meets the token minimum, monitor cache utilization, and keep sensitive data out of the cached segment. By doing so, you can cut token costs, lower latency, and maintain consistent agent behavior across long sessions and multiple sub‑agents.