The latest OpenChoreo release ships a dedicated OpenTelemetry Collector distribution that handles two observability streams out of the box: end‑to‑end request tracing and long‑term Kubernetes event storage. By embedding the collector in each data plane and in the central observability plane, the platform eliminates the need for ad‑hoc integrations and lets engineers swap back‑ends via Helm modules.
Distributed Tracing Pipeline
Application code is instrumented with the standard OpenTelemetry SDK and emits spans over OTLP. A local collector instance in every data plane receives those spans via the otlp receiver, ensuring that traffic never leaves the originating cluster before processing. The k8sattributes processor enriches each span with pod metadata, and because OpenChoreo labels every workload, the spans inherit project, component, and environment identifiers automatically.
Sampling decisions are deferred to the tail_sampling processor, which evaluates the full trace before discarding or keeping it. Enriched traces are then exported either to a central collector in the observability plane or directly to a cloud backend using one of the supported exporters (googlecloud, awsxray, azuremonitor, otlphttp). Authentication details are supplied through the collector’s auth extensions, keeping credentials out of application code. The modular design means swapping from OpenSearch to AWS X‑Ray is a matter of installing a different Helm chart, not rewriting pipelines.
Persistent Kubernetes Event Capture
Kubernetes events, which often explain resource‑level incidents, are short‑lived by default. OpenChoreo’s custom collector runs a single replica of the k8s_events receiver, guaranteeing each event is recorded once. The bespoke k8seventenrich processor looks up the referenced object (Pod, Deployment, etc.) and copies its labels, annotations, and owner reference into the event as resource attributes (e.g., k8s.object.label.*). These lookups use in‑memory informer caches, so the enrichment path adds no extra API‑server load.
After enrichment, a batch processor forwards events to the configured storage backend. Because events retain the same label schema as traces, they can be queried alongside other observability data, providing a unified view of incidents that span both application and platform layers.
Operational and Security Implications
From an operations standpoint, the collector’s “plane‑local” deployment isolates processing to the workload’s cluster, reducing cross‑cluster latency and limiting blast radius. The hexagonal architecture of the Observer component decouples inbound APIs (logs, metrics, traces) from storage adapters, allowing platform teams to adopt or replace back‑ends without code changes. Credential handling is confined to the collector’s auth extensions, which centralises secret management and avoids embedding secrets in application containers.
Security considerations include ensuring the collector’s service account has read access to pod metadata and event informers, but no write privileges beyond what is required for exporting data. Because the collector runs as a single replica for event capture, high availability for that specific path must be addressed at the platform level if loss of events is unacceptable.
Related CloudNinjas coverage: hands-on guides.
What This Means For Practitioners
Engineers building on OpenChoreo can now rely on a single, configurable collector for both tracing and event observability, simplifying deployment pipelines and reducing the number of moving parts. Evaluate the existing Helm modules to select a storage backend that matches your compliance and performance needs, and verify that the collector’s service account scopes are minimal. Finally, monitor the collector’s health metrics to ensure that local processing does not become a bottleneck as workload volume grows.