EXL replaced a manual, 100‑minute‑per‑case review process with an end‑to‑end medical IDP pipeline that runs on AWS, pairing a template‑agnostic document processor (Xtrakto.AI) with a domain‑specific large language model (EXL Insurance LLM). The change shifts the bottleneck from human effort to a managed, scalable set of cloud services, which directly impacts AI engineers, platform engineers, SREs, and security teams.
Architecture Overview
The pipeline lives entirely within a single AWS Region and follows an eleven‑step flow that begins with a secure API call and ends with a structured response. Core services include:
Amazon API Gateway– exposes ingestion and result delivery endpoints.Amazon Cognito– authenticates and authorizes each request.AWS Step Functions– orchestrates the sequence of sub‑tasks.Amazon TextractandAWS Lambda– perform initial document preprocessing, such as OCR and format normalization.Amazon SageMaker– hosts the fine‑tuned EXL Insurance LLM, providing isolated training environments and real‑time inference endpoints that scale with claim volume.Amazon Bedrock– supplies on‑demand access to general‑purpose foundation models for tasks that do not require domain‑specific reasoning.
All components are bound by AWS Identity and Access Management (IAM) policies, ensuring that only authorized roles can invoke services that handle protected health information (PHI).
Implementation Highlights for Engineers
AI engineers benefit from a clear separation between model development and production inference. SageMaker’s managed training environment supports multi‑GPU fine‑tuning of the Insurance LLM, while experiment isolation prevents accidental cross‑contamination of data. In production, SageMaker endpoints can be auto‑scaled based on request rates, eliminating the need for custom autoscaling logic.
Platform engineers can reuse the same Bedrock API surface to call either the fine‑tuned LLM or a general‑purpose model, reducing operational overhead of managing multiple model runtimes. The template‑agnostic nature of Xtrakto.AI means that new document types can be introduced without redefining extraction templates, simplifying CI/CD pipelines for document schema updates.
SREs gain a deterministic orchestration layer via Step Functions, which provides built‑in retry, timeout, and state‑visualization capabilities. Each step can be instrumented with CloudWatch metrics, enabling alerting on latency spikes or error rates without additional code.
Operational and Security Implications
Running the entire workflow inside a single Region limits data residency concerns and simplifies compliance audits. IAM scoping, combined with Cognito authentication, creates a clear boundary for who can invoke the API and which downstream services they may access. Because PHI traverses Textract, Lambda, SageMaker, and Bedrock, each service must be granted the minimum required permissions, reinforcing the principle of least privilege.
From an operational standpoint, the separation of a dedicated model‑development environment allows continuous improvement of the Insurance LLM without impacting live inference traffic. However, practitioners must monitor model drift and versioning, as updates to the fine‑tuned model could affect downstream summarization accuracy.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Engineers should evaluate the trade‑offs of using managed services versus self‑hosted alternatives, especially regarding cost of multi‑GPU training and Bedrock usage. Security teams must verify IAM policies and Cognito configurations to ensure PHI never leaves the protected region. SREs should instrument Step Functions and SageMaker endpoints for latency and error monitoring, and set up automated roll‑backs for model releases that degrade performance. Finally, ongoing governance of LLM outputs—such as traceability of reasoning and validation against medical coding standards—will be essential to maintain regulatory compliance.
