Standard Bank Group replaced a manual, document‑heavy review process with an automated pipeline that scores every generative AI use case against its own policies and international AI standards. For engineers, platform owners, and security teams this means the evidence needed for risk approval is collected, tested, and reported without human‑driven paperwork, freeing time for development and reducing the chance of inconsistent assessments.
Architecture Overview
The pipeline lives entirely inside the bank’s AWS account. Core components are:
- AWS Well‑Architected Tool – hosts a custom lens with roughly 40 risk questions.
- AWS Fargate – runs a pre‑assessment agent that scans the code repository and documentation, maps files to lens questions, and pre‑populates answers with links to the evidence.
- AWS Step Functions – orchestrates the four‑step workflow.
- Amazon Bedrock – provides a foundation model that acts as an “LLM‑as‑a‑judge”, evaluating the collected evidence against the defined controls.
- Amazon S3 – stores all artifacts (system docs, policy docs, source code, prompts, and live call logs) used by the pipeline.
IAM controls access to the Bedrock model and the S3 bucket, ensuring the same identity management used for other workloads applies here. The model does not retain prompts or outputs, and the bank can swap the judge model by changing a configuration value rather than rebuilding the pipeline.
Implementation Details
The workflow proceeds through four distinct steps:
- Evidence collection – The Fargate‑based agent crawls the repository, identifies relevant files, and fills the Well‑Architected lens. Teams review and correct the auto‑filled answers before proceeding.
- Artifact examination and testing – The pipeline consumes four evidence categories: system documentation, policy documents, source code plus the system prompt, and live API calls. It validates that each artifact contains the required information and runs functional tests against the live application.
- Control evaluation – The Bedrock judge compares the gathered evidence to the risk criteria defined in the lens, producing a compliance score for each control.
- Reporting – Results are formatted into a standard report and pushed to a dashboard that shows what is compliant, what needs mitigation, and overall risk posture.
Because the pipeline runs in the bank’s own account, data never leaves the controlled environment, and the same IAM policies that protect production workloads also protect the assessment data.
Operational and Security Implications
Automating evidence gathering reduces the manual effort required for each new generative AI project, turning a days‑long review into a repeatable, minutes‑long run. Consistent evidence collection also improves auditability; every use case follows the same evidence‑mapping logic, making it easier to trace decisions back to source artifacts.
From a security perspective, the design isolates the assessment data in a dedicated S3 bucket and leverages IAM for model access, limiting exposure. The use of Bedrock’s “no‑training‑on‑prompts” guarantee aligns with the bank’s data‑privacy requirements. However, the pipeline still depends on the correctness of the pre‑assessment agent and the Bedrock judge, so teams should monitor for false‑positive or false‑negative evaluations and have a manual override path for edge cases.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Engineers should consider adopting a similar pattern when their organization needs repeatable AI governance:
- Leverage existing compliance frameworks (e.g., Well‑Architected lenses) to avoid building new policy stores.
- Use serverless compute (Fargate, Step Functions) to keep the pipeline lightweight and easily versioned.
- Choose a foundation model that can be called via a single API and that respects data‑privacy guarantees.
- Instrument the pipeline to emit dashboards and reports that feed directly into existing approval gates.
- Plan for periodic review of the judge model and the evidence‑mapping logic to keep pace with rapid changes in generative AI tooling.
By automating the evidence lifecycle, teams can maintain rigorous risk standards without slowing the delivery of generative AI capabilities.

