ZS Associates replaced a loosely governed SageMaker setup with a secure SageMaker platform that isolates each tenant, enforces encryption, and removes direct internet access. Practitioners care because the design demonstrates how to grant data scientists self‑service analytics while satisfying strict healthcare compliance and controlling cost.
Secure SageMaker platform architecture
The solution adopts a multi‑tenant model where each business unit receives its own Amazon SageMaker domain. Isolation is achieved through dedicated Amazon EFS volumes, distinct IAM roles, and configurable VPC settings per domain. A three‑tier role hierarchy—Domain Execution Roles as defaults, Studio User Roles for fine‑grained overrides, and Space Execution Roles for shared workspaces—applies the principle of least privilege across the environment.
Implementation details for compliance
All deployments run in internet‑free mode; connectivity to required AWS services is provided via VPC endpoints. Package management is routed through JFrog Artifactory, which links upward to corporate repositories and performs real‑time scanning to block unauthorized or tampered artifacts. Encryption at rest is enabled by default using AWS KMS for EFS, S3, ECR, and CodeCommit repositories. Continuous security monitoring leverages CrowdStrike for host‑level threat detection and Splunk for log aggregation, while CloudTrail records every API call to support audit trails.
Because SageMaker lacks native backup, a custom lifecycle configuration periodically copies user notebooks and scripts to Amazon S3, giving each user a resilient workspace without manual effort. Cost‑control policies restrict instance selection to a curated list of smaller types; requests for larger instances must be approved through an allow‑list process. Auto‑shutdown rules automatically terminate idle resources, preventing waste.
Operational impact and cost control
The platform now supports over 1,000 daily active users across more than 200 SageMaker domains. Tag‑based cost allocation enables precise charge‑back by team and project, feeding into AWS Cost Explorer for visibility. Monthly SageMaker spend exceeds $50 K, and ZS applies SageMaker Savings Plans to reduce that expense. The combined automation of compliance checks, backup, and resource lifecycle has eliminated manual overhead and reduced idle‑resource spend.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
Teams looking to expose ML capabilities in regulated environments should evaluate a domain‑per‑tenant approach, enforce VPC‑only connectivity, and adopt a layered IAM role model. Integrating artifact scanning and automated lifecycle hooks can close compliance gaps without sacrificing developer agility. Finally, systematic tagging and Savings Plan enrollment are essential for turning a secure, multi‑tenant SageMaker deployment into a cost‑effective service.
