We replaced a collection of bespoke data‑access adapters with a single, serverless Model Context Protocol gateway running on Amazon ECS Fargate. The new design lets any MCP‑compatible AI agent query the enterprise data lake through a shared endpoint, while a JSON metadata file drives dataset discovery and schema exposure.
Why the Shift to a Metadata‑Driven MCP Gateway
Previously each AI agent embedded its own SQL generation, schema knowledge, and connection handling. Adding or changing a dataset required custom code, individual permission checks, and a multi‑day rollout. By moving to a metadata catalog that the MCP server reads at runtime, the integration point becomes dataset‑agnostic: a new JSON file is sufficient to expose a table, and schema changes propagate without redeploying agents. Centralizing permission enforcement at the server layer also provides a single audit point for data access decisions.
Key Architectural Components
- Client entry point: DNS resolves via Amazon Route 53 to an Internet‑facing Application Load Balancer (ALB) protected by AWS WAF. The ALB forwards traffic to Amazon Cognito for authentication and user attribute enrichment.
- Session layer: Authenticated requests enter a private subnet where a Fargate task runs the client component. Session state is persisted in Amazon DynamoDB.
- MCP server layer: Another Fargate task hosts the MCP server built with
FastMCP. The server invokes Amazon Bedrock models, applying Bedrock Guardrails to every interaction. - Tool backends: SQL queries are executed by Amazon Athena or Amazon Redshift against the data lake stored in Amazon S3. Semantic retrieval uses Amazon Bedrock Knowledge Bases, while data ingestion and catalog updates are performed by AWS Lambda functions.
The architecture isolates each concern into its own scalable layer, allowing independent scaling of client traffic, MCP request handling, and data‑layer query execution.
Operational and Security Considerations
Running the gateway on Fargate eliminates the need to manage EC2 instances, but it also introduces new operational metrics: task concurrency, CPU/memory limits, and container image versioning become the primary levers for capacity planning. The metadata catalog must be kept in sync with the data lake; the pipeline that writes JSON files should be monitored for failures, as a missing file prevents dataset exposure.
From a security perspective, centralizing access control at the MCP server simplifies policy management, yet it also concentrates risk. Practitioners should ensure that Cognito authentication, WAF rules, and IAM roles for the Fargate tasks are tightly scoped. Logging from the ALB, Cognito, and the MCP server should be aggregated for auditability. Because the server forwards queries to Athena, Redshift, and Bedrock, the IAM roles attached to the server task must include only the permissions required for the specific data sources it accesses.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Adopting a serverless MCP gateway reduces integration overhead and aligns data‑access permissions with a single enforcement point. Engineers should evaluate their existing AI agents for compatibility with the Model Context Protocol, migrate metadata definitions to the JSON format, and configure Fargate task sizing to match expected query bursts. Ongoing operational focus will be on monitoring task health, ensuring the metadata pipeline stays current, and tightening the IAM role boundaries that the gateway uses to reach Athena, Redshift, and Bedrock services.
