Live
Local Artifacts Support Lets Workers Test Repo Workflows Before DeployCo‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI ClusterScaling AI Agents on Kubernetes: New CNCF Projects and Operational ImplicationsGitHub Copilot model deprecation forces updates to AI‑assisted development pipelinesGitHub Copilot expands Enterprise Server access and adds new AI models – practical impact for engineersArchitecting Agentic Customer Engagement with Amazon Bedrock AgentCoreBuilding Multi‑Tenant Memory‑Aware Agents with Bedrock Agentic Retrieve and Managed Knowledge BaseBulk‑Start AI Scan for Up to 20 GitHub Repos: What Engineers Need to KnowLocal Artifacts Support Lets Workers Test Repo Workflows Before DeployCo‑Packaged Optics Cut Power Use and Latency in Lambda’s 10k‑GPU AI ClusterScaling AI Agents on Kubernetes: New CNCF Projects and Operational ImplicationsGitHub Copilot model deprecation forces updates to AI‑assisted development pipelinesGitHub Copilot expands Enterprise Server access and adds new AI models – practical impact for engineersArchitecting Agentic Customer Engagement with Amazon Bedrock AgentCoreBuilding Multi‑Tenant Memory‑Aware Agents with Bedrock Agentic Retrieve and Managed Knowledge BaseBulk‑Start AI Scan for Up to 20 GitHub Repos: What Engineers Need to Know
Kubernetes

Scaling AI Agents on Kubernetes: New CNCF Projects and Operational Implications

AI SummaryPowered by AI

KubeCon introduced OpenShell and Agent Substrate as CNCF Sandbox projects to address the mismatch between human‑centric systems and the workload patterns of AI agents. Practitioners must adapt architecture, operations, and security practices to support agent‑scale workloads without over‑provisioning resources.

At KubeCon + CloudNativeCon NA, the CNCF announced two sandbox projects—OpenShell (contributed by Nvidia) and Agent Substrate (contributed by Google)—that target the unique speed and usage patterns of AI agents. Existing cloud‑native stacks were designed for human‑driven traffic, and Bryce of the CNCF warned that those systems are not yet resilient or scalable enough for the continuous, on‑demand interactions agents generate. Engineers who build, run, or secure AI workloads need to understand how these projects change the way agents are isolated, managed, and observed on Kubernetes.

AI agents and the need for new control planes

AI agents differ from traditional services in that they may sit idle for long periods, then be called into action at a moment’s notice. This bursty, latency‑sensitive pattern stresses scheduling, resource allocation, and policy enforcement. OpenShell introduces a policy layer that can restrict which files, network endpoints, credentials, and model endpoints an agent may access. Agent Substrate complements this by providing per‑agent isolated environments on Kubernetes, with built‑in support for suspending idle sessions and resuming them on demand. Both projects aim to embed security and control at the infrastructure level rather than relying on application‑specific checks.

Operational considerations for agent lifecycle management

Because agents are not continuously active, keeping them running full‑time wastes CPU cycles. Agent Substrate’s state‑save capability allows the platform to spin down an idle agent, reclaim compute, and later restore the session on an available worker. Practitioners should evaluate how to integrate this suspend‑resume flow with existing CI/CD pipelines and monitoring stacks. The approach also raises questions about state consistency, checkpoint granularity, and the latency of warm‑up when an agent is re‑activated.

Security workload and vulnerability reporting

The conference highlighted a surge in vulnerability reports generated by AI tools. Open source maintainers are receiving many duplicate or unverified findings, yet they must still triage each report. This increased noise can strain security operations and requires better filtering or attribution mechanisms. While the source does not detail specific controls, the implication is that teams should allocate resources for handling AI‑driven reporting and consider how OpenShell’s policy enforcement might reduce exposure by limiting agent access to sensitive assets.

Related CloudNinjas coverage: hands-on guides.

What This Means For Practitioners

Engineers should start by reviewing the OpenShell policy model to see how it maps to existing RBAC or admission‑controller configurations. If agents are being deployed on shared clusters, adopting Agent Substrate can provide isolation and a built‑in idle‑state mechanism, reducing unnecessary compute spend. Monitoring setups need to capture agent‑specific metrics—such as suspend/resume latency and policy violation attempts—to maintain observability at the agent level. Finally, security teams must prepare for higher volumes of AI‑generated vulnerability reports and consider process adjustments to handle duplicate or low‑confidence findings without compromising response quality.

Originally published atThe New Stack