The Web Search API entered beta on Cloudflare and is now exposed through the AI Gateway service. It lets AI workloads retrieve live web results from three supported providers, eliminating reliance on static model knowledge and exposing the request path to existing gateway observability and billing. Engineers can invoke the endpoint via a standard REST request or the AI binding inside Workers, and they may supply their own provider key if desired.
What Changed
Cloudflare added a new beta endpoint under the AI Gateway namespace that forwards search queries to external providers. The initial provider set includes Ceramic.ai, Exa, and Linkup, each promising zero data retention for requests that pass through Cloudflare. The service uses the same credit‑based billing model as other AI Gateway calls, applying the provider’s listed API price without an additional Cloudflare surcharge.
Why It Matters for Engineers
Live web results enable AI agents to ground responses in current information, reducing hallucinations that stem from outdated training data. Because the calls travel through the AI Gateway, they appear in existing logs, making it straightforward to monitor latency, error rates, and cost alongside other model invocations. The zero‑retention promise simplifies compliance for workloads that must avoid persisting query content.
Architectural and Operational Implications
Integrating the Web Search API requires only a change in the request target. Existing AI Gateway configurations—such as gateway IDs, credit pools, and log aggregation—continue to apply. Practitioners should consider the following operational points:
- Provider selection is a request‑time parameter; switching providers does not require redeploying the gateway.
- Each provider’s pricing is applied at call time, so cost monitoring must account for variable per‑query rates.
- Custom provider keys can be supplied, which means secret management must be added to the deployment pipeline if the default Cloudflare‑managed keys are not used.
- Rate limits and result limits (e.g., the
limitfield) are enforced by the downstream provider, not by Cloudflare.
Provider selection flexibility
Because the provider field is part of the JSON payload, teams can experiment with different sources in staging or A/B tests without changing infrastructure. This flexibility supports gradual migration to a preferred provider based on relevance, latency, or cost.
Cost and usage tracking
All search calls are billed to the AI Gateway credit pool at the provider’s list price, and they appear in the same gateway logs used for model calls. Engineers should extend existing log‑based dashboards to include the new /ai/websearch/ endpoint, enabling alerts on unexpected spend spikes.
Secret management for custom keys
If a team brings its own provider API key, that secret must be stored in a secure runtime store (for example, Workers Secrets or an external vault) and injected only at execution time. Rotating the key requires updating the secret without redeploying the worker code.
Example REST call (curl) demonstrates the required headers and JSON payload:
curl https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/websearch/ \
--request POST \
--header "Authorization: Bearer $API_TOKEN" \
--header "Content-Type: application/json" \
--data '{
"query": "What are some fun things to do in Salt Lake City as fall approaches?",
"provider": "ceramic",
"limit": 5,
"options": { "gateway": { "id": "default" } }
}'Within a Cloudflare Worker the same operation uses the AI binding:
const response = await env.AI.websearch({
gatewayId: "default",
query: "What are some fun things to do in Salt Lake City as fall approaches?",
provider: "exa",
limit: 5,
});
const results = await response.json();Security and Data‑Retention Considerations
All three providers commit to zero data retention for requests that traverse Cloudflare, which reduces the attack surface associated with stored query content. The providers also adhere to Cloudflare’s verified‑bot crawling standards, meaning they are expected to respect robots.txt and other bot policies. Nevertheless, practitioners should still treat the provider API key as a secret and restrict its exposure to the runtime environment that performs the search.
Compliance impact
Zero‑retention aligns with many data‑privacy regulations that prohibit long‑term storage of user‑generated queries. Teams can document the end‑to‑end flow—client → AI Gateway → provider—as a bounded process that does not retain payloads beyond the request lifecycle.
Related CloudNinjas coverage: hands-on guides.
What This Means For Practitioners
Adopt the Web Search API where up‑to‑date information is required, and leverage the existing AI Gateway pipeline for observability and cost tracking. Evaluate each provider’s response quality, latency, and pricing before committing to a default. Ensure secret handling for any custom provider keys and monitor gateway logs for unexpected traffic patterns that could indicate misuse. Keep an eye on the beta status for future provider additions, pricing updates, or changes to the zero‑retention guarantee.
