Integrating Cloudflare D1 as a Data Source for AI Search (beta)

AI Search has been extended to pull data directly from a D1 database. By supplying a read‑only SQL query, each returned row is turned into a separate document and kept up‑to‑date through scheduled synchronisation. Practitioners can provision these instances via the dashboard, the REST API, or a Workers binding, eliminating the need for external ETL steps.

Configuring a D1‑backed AI Search instance

The service expects a JSON payload that identifies the instance, declares the source type as d1, and provides the database identifier, a service token, and the query to execute. The following curl example demonstrates the required request:

curl -X POST "https://api.cloudflare.com/client/v4/accounts//ai-search/namespaces/default/instances" \
  -H "Authorization: Bearer " \
  -H "Content-Type: application/json" \
  --data '{
    "id": "help-center",
    "type": "d1",
    "source": "",
    "token_id": "",
    "source_params": {
      "d1": {
        "query": "SELECT article_id AS id, title, content, category AS meta_category\nFROM articles\nWHERE published = 1"
      }
    }
  }'

When the instance is created, AI Search runs the query, creates a document per row, and schedules periodic re‑runs to capture inserts, updates, or deletions.

Operational considerations

• Sync frequency: The service performs scheduled syncs, but the interval is not configurable in the source. Teams should monitor index freshness against their data change rate.
• Query shape: Only columns needed for search should be selected. The example maps article_id to id and uses category as a metadata field, illustrating how to shape the document schema.
• Performance impact: Each sync re‑executes the full query. Large tables may increase D1 read costs and latency; consider filtering or pagination if appropriate.
• Beta status: As a beta feature, behaviour may evolve. Deploy to non‑critical environments first and track release notes for breaking changes.

Security considerations

• Service token protection: The token_id grants AI Search read access to the specified D1 database. Store the token securely and rotate it according to your secret‑management policy.
• Read‑only query: The integration only issues the supplied SELECT statement, limiting exposure to write operations. Nevertheless, ensure the query does not return sensitive columns unintentionally.
• Data exposure via indexing: Indexed documents become searchable through AI Search APIs. Review the fields included in the index to avoid leaking confidential information.

Related CloudNinjas coverage: hands-on guides.

What This Means For Practitioners

Adopting the D1 data source removes a custom ingestion layer for relational content, but it introduces new responsibilities around query design, token hygiene, and sync monitoring. Evaluate the size and change frequency of your tables, restrict indexed fields to what is needed for search, and treat the service token as a privileged credential. Until the feature graduates from beta, plan for a fallback ingestion path in case of regressions.