Live
Cloudflare Images Analytics Dashboard Adds 30‑Day Delivery VisibilityAmazon ECS Adds Native Blue/Green, Canary, and Linear Deployments for Safer Container RolloutsGemini agentic tools add LLM‑driven automation to Partner Marketing StudioGitHub Actions makes RHEL 9/10 runner images generally available for larger workloadsIntegrating Decider Models to Reduce Latency and Cost in AI Agent WorkflowsAWS Emulator Enables Local Testing of 200+ Services, but Lacks IAM ValidationSageMaker AI monitoring adds automatic latency drift, idle GPU, and configuration gap detection for inference endpointsEmbedding Proactive Governance into Enterprise‑Scale Agentic AICloudflare Images Analytics Dashboard Adds 30‑Day Delivery VisibilityAmazon ECS Adds Native Blue/Green, Canary, and Linear Deployments for Safer Container RolloutsGemini agentic tools add LLM‑driven automation to Partner Marketing StudioGitHub Actions makes RHEL 9/10 runner images generally available for larger workloadsIntegrating Decider Models to Reduce Latency and Cost in AI Agent WorkflowsAWS Emulator Enables Local Testing of 200+ Services, but Lacks IAM ValidationSageMaker AI monitoring adds automatic latency drift, idle GPU, and configuration gap detection for inference endpointsEmbedding Proactive Governance into Enterprise‑Scale Agentic AI
Kubernetes

Kyverno 1.19.2 eliminates per‑policy object conversion, slashing admission latency

AI SummaryPowered by AI

Kyverno 1.19.2 moves the JSON‑to‑unstructured conversion out of the per‑policy loop, allocating the request object only once per admission request. This change cuts allocation overhead and latency, which matters for clusters with many policies or large objects.

The latest Kyverno release (1.19.2) changes the way admission requests are prepared for CEL evaluation: the raw API payload is now converted to the unstructured map a single time, instead of once for every matching policy. Practitioners see a dramatic drop in memory allocations and per‑request latency, especially when dozens of policies apply to a single object.

Root Cause of the Original Overhead

When an admission webhook receives a AdmissionRequest, Kyverno must translate the raw JSON into a map that CEL can read. The original implementation performed this conversion inside the loop that iterates over each matched policy. Because the same object is processed repeatedly, the cost grew linearly with the number of policies (O(policies)). Profiling showed that roughly 86 % of all allocations were spent in the ConvertObjectToUnstructured function, while the actual expression evaluation contributed almost nothing.

Implementation of the Fix

The fix extracts the conversion step from the per‑policy loop. Before iterating over policies, Kyverno builds a single CEL request value and reuses it for every policy evaluation. To preserve existing semantics, the engine temporarily blanks out the object field during conversion, walks only the cheap fields, and then splices the original object back in. This approach avoids a full reimplementation while guaranteeing identical results. Separate validation and mutation engines each retain their historic contract, and a suite of equivalence tests compares the new output field‑by‑field against the reference implementation, causing the build to fail on any drift.

Operational Impact

Benchmarks that call the engine directly with a ~50 KB Pod illustrate the effect. With one policy, allocations drop from 2,327 to 854 bytes and latency from 0.31 ms to 0.10 ms. At 64 policies, allocations shrink from 148,131 to 8,672 bytes and latency from 20.01 ms to 0.36 ms for the validating engine. The mutating engine shows a similar pattern, falling from 258,943 to 52,689 allocations and from 10.59 ms to 2.97 ms at 64 policies. The residual growth after the fix corresponds to the genuine cost of evaluating CEL expressions, which is the work users intentionally added. The image‑verification engine, which used its own copy of the conversion code, receives the same memory‑usage benefit, though its overall speed is dominated by network calls.

Related CloudNinjas coverage: hands-on guides.

What This Means For Practitioners

  • Upgrade any Kyverno deployment to version 1.19.2 or later to reap the allocation and latency improvements.
  • Re‑measure admission latency after upgrade; the benchmark excludes webhook round‑trip time, so end‑to‑end latency will improve proportionally.
  • If you run a large number of policies or admit large objects (e.g., multi‑container Pods), expect a noticeable reduction in CPU pressure on the webhook pod.
  • Validate that custom policies still behave as intended; the equivalence tests in Kyverno’s CI pipeline aim to catch regressions, but a quick smoke test in your environment is prudent.
  • Consider that the per‑policy serialization cost is now gone, so scaling policy count no longer incurs hidden performance penalties.
Originally published atCNCF