Cutting Input Token Costs and Latency with Amazon Bedrock Prompt Caching

Amazon Bedrock now offers a prompt‑caching feature that lets you store a snapshot of the processed portion of a request and reuse it on later calls that contain the same prefix. For teams that repeatedly send large documents or system prompts to foundation models, this change can slash input‑token spend by up to 90 % and cut the time‑to‑first‑token, directly improving both cost efficiency and latency.

How Prompt Caching Works

When a request to the Converse API includes a cachePoint marker, Bedrock checks whether the tokens that appear before the marker match an existing cache entry. A match (a cache hit) causes the model to skip re‑processing those tokens and start generation from the cached state. If no entry exists (a cache miss), the model processes the full request and writes the resulting state to the cache for future hits.

Four operational parameters control the behavior:

1. Cache scope: Entries are isolated to the AWS account and Region that created them.
2. Token thresholds: A checkpoint must contain a minimum number of tokens before Bedrock will cache it. Anthropic Claude Sonnet 4.5/4.6 require at least 1,024 tokens; Opus models require at least 4,096 tokens.
3. Time‑to‑live (TTL): The default TTL is five minutes; selected models allow up to one hour.
4. Model‑agnostic syntax: The cachePoint syntax is identical across supported families, including Claude and Amazon Nova.

Cost and Performance Implications

Prompt caching introduces three token categories beyond the standard input and output tokens:

• cacheWriteInputTokens: Tokens written on the first request; billed 25 % higher than standard input.
• cacheReadInputTokens: Tokens read on subsequent requests; billed 90 % lower than standard input.
• cacheWriteInputTokens with a one‑hour TTL: Charged at double the standard input rate.

Because the read cost is dramatically reduced, workloads that reuse the same context see an overall input‑token saving of roughly 75 %. For example, a 10,000‑token contract followed by ten questions incurs a full‑price write on the first call and nine reads at the reduced rate, delivering the quoted savings as long as the reads occur within the TTL window.

Design Patterns and Operational Considerations

Practitioners can map the six scenarios described by the Bedrock team to concrete architecture decisions:

• Message content caching: Store long reference documents once and reuse them for multiple question‑answer turns.
• System prompt caching: Cache persona or instruction blocks that remain constant across sessions.
• Tool definition caching: Cache schema definitions for agentic tool calls, avoiding repeated transmission.
• Mixed TTL caching: Assign longer TTLs to rarely changing data (e.g., policy documents) and shorter TTLs to more volatile content.
• Tenant isolation: In multi‑tenant services, separate cache namespaces per tenant to prevent cross‑tenant data leakage.
• LangChain integration: Leverage the LangChain framework’s built‑in support to inject cachePoint markers automatically.

From an operations perspective, monitoring cache hit ratios and TTL expirations can reveal whether the chosen thresholds align with real traffic patterns. Adjusting the token threshold or TTL per model may be necessary to balance cache effectiveness against the higher write cost for longer TTLs.

Related CloudNinjas coverage: AWS.

What This Means For Practitioners

Adopt cachePoint in any prompt that contains a stable prefix—documents, system prompts, or tool schemas—to capture up to 90 % of input‑token spend on repeat calls. Verify that the prefix meets the model‑specific token minimum, and configure TTLs that match your usage window to avoid unnecessary cache writes. Track hit rates and adjust cache lifetimes to keep the 75 % overall savings realistic. Finally, ensure cache isolation per account, region, and tenant to maintain data separation without adding extra security controls beyond what Bedrock already scopes.