Integrating an AI‑driven triage agent into EKS CI/CD with Bedrock and AI‑DLC

Rescale swapped a manual, on‑call‑driven process for classifying nightly end‑to‑end test failures with an AI triage agent that executes as a step in their Argo Workflows CI/CD pipeline on Amazon EKS. Practitioners care because the agent eliminates hours of repetitive investigation, reduces coordination friction, and demonstrates a concrete way to embed generative AI into production pipelines.

Why the existing workflow was a bottleneck

Each night the platform runs hundreds of Playwright tests. When failures appear, engineers must pull logs, search historical patterns, and locate the owning code repository before they can file a bug or route the issue. Because the test suite gates weekly production releases, unresolved failures can stall deployments and force multiple teams into ad‑hoc troubleshooting sessions.

AI triage agent architecture

The agent is launched as a pod by Argo Workflows after a test run reports failures. Its workflow consists of four sequential actions:

1. Call Claude Opus 4.8 on Amazon Bedrock to assign each failure to one of four root‑cause categories: infrastructure issue, code bug, flaky test, or environment problem.
2. Query a knowledge base for prior failure patterns that match the current symptoms, providing additional context for the classification.
3. Use the Model Context Protocol to retrieve relevant pod logs, giving runtime evidence that supports the model’s decision.
4. Emit a structured remediation report that lists file paths, function names, and suggested fixes. The report is fed directly into the team’s existing reporting pipeline.

The entire sequence completes in seconds, allowing downstream automation to proceed without human delay.

Implementation with AI‑DLC

Rescale adopted the open‑source AI‑DLC methodology to keep the development pace aligned with a five‑day sprint to staging and a few weeks to production. Two collaborative session types drove the effort:

• Mob Elaboration: the team defined the classification categories, knowledge‑base schema, and report format together, while the AI generated draft specifications.
• Mob Construction: the AI produced code snippets for the Bedrock invocation, MCP log retrieval, and Argo task definitions. Engineers reviewed each snippet in real time, approving or correcting before the AI moved forward.

This approach ensured that the AI’s understanding matched the team’s intent, avoiding hidden assumptions that often surface in asynchronous hand‑offs.

Operational impact and considerations

Deploying the agent introduced several practical implications:

• Latency dropped from hours of manual triage to sub‑minute automated classification, directly shortening the release window.
• Human review remains a gate before any remediation is applied, preserving safety while still gaining speed.
• Reliance on Claude Opus 4.8 means that model availability and cost must be monitored as part of the service‑level budget.
• The knowledge base and MCP integrations require consistent schema management; changes to log formats or failure metadata will need coordinated updates.

Because the agent runs inside the same EKS cluster as the test suite, it inherits the cluster’s security posture and can be governed by existing pod‑security policies.

Related CloudNinjas coverage: AWS.

What This Means For Practitioners

Teams facing repetitive classification or routing tasks can replicate this pattern by:

1. Identifying a well‑scoped problem where the hard part is translating intent to code.
2. Adopting AI‑DLC sessions to keep AI output aligned with human expectations.
3. Embedding a Bedrock model call as a microservice step within an existing CI/CD workflow.
4. Ensuring a human validation checkpoint before any automated remediation is enacted.

By treating the AI component as another pipeline stage rather than a black‑box replacement, engineers retain control while harvesting the speed gains of generative models.