AWS has added a pattern that turns the investigation output of the DevOps Agent into an automated remediation workflow built on EventBridge, Lambda Durable Functions, and Bedrock. Engineers can now move from root‑cause analysis to validated fixes without manual scripting, while still requiring explicit approval for any changes that modify production resources.
Automated remediation workflow overview
The flow begins when the DevOps Agent finishes an incident investigation and publishes an event that contains symptoms, findings, and a root‑cause analysis. EventBridge captures this event and invokes the devops-agent-trigger Lambda function, which packages the investigation summary and starts the devops-agent-remediation-durable durable function. The durable function forwards the context to Amazon Bedrock, which parses the findings and selects remediation tools from a curated allowlist of approved Lambda functions (named devops-agent-lambda-tool). Bedrock then proposes concrete remediation actions. Read‑only tools are executed automatically; any tool that would mutate infrastructure causes the durable function to pause and wait for a human approval signal before proceeding. After approval, the selected tools run against the target resources, completing the remediation loop.
Implementation steps
- Configure the DevOps Agent to emit an
investigation-completedevent to EventBridge. - Create an EventBridge rule that targets the
devops-agent-triggerLambda function. - In the trigger function, format the investigation payload and start the
devops-agent-remediation-durableorchestrator using the Lambda Durable Functions API. - Within the durable function, call Bedrock with the investigation context. Bedrock returns a list of applicable
devops-agent-lambda-toolfunctions drawn from the allowlist. - Execute each tool that is classified as read‑only. For tools that would change state, emit a waiting event and expose a manual approval checkpoint (for example, via an SNS notification or a custom approval UI).
- Upon receipt of the approval signal, resume the durable function and invoke the mutating tools.
- Collect tool results, feed them back to Bedrock if further iteration is needed, and finally log the remediation outcome.
Operational and security implications
Lambda Durable Functions provide built‑in checkpointing and state persistence, allowing the orchestration to survive restarts, timeouts, or long‑running tasks without additional infrastructure. This resilience reduces the risk of orphaned remediation attempts and simplifies error handling.
Because Bedrock can only select from the pre‑approved devops-agent-lambda-tool set, the attack surface is limited to functions that have been vetted for purpose and scope. This allowlist model is a practical control that keeps automated actions within defined boundaries.
The split between read‑only and mutating actions enforces a manual review point for any change that could affect production state. Practitioners must design the approval mechanism to be auditable and to integrate with existing change‑management processes.
All events and function invocations are recorded in CloudWatch Logs and can be correlated with the original DevOps Agent investigation, providing a traceable audit trail for compliance or post‑mortem analysis.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Adopting this pattern lets on‑call teams close incidents faster by automating the repetitive parts of remediation while preserving human oversight for risky changes. Teams should inventory existing Lambda tools, define an allowlist, and implement a reliable approval workflow before enabling the automated path. Monitoring the durable function’s state and logging will be essential to ensure visibility and to detect any unexpected pauses or failures.
