Deepgram has added two observability extensions for its speech‑to‑text and text‑to‑speech models running on Amazon SageMaker AI. The extensions publish billing‑aligned usage metrics directly to CloudWatch and expose engine‑level Prometheus/OpenTelemetry data without requiring sidecars, agents, or extra IAM permissions. Engineers gain fine‑grained cost visibility and GPU performance data while keeping all traffic inside their own AWS account.
Deepgram Enhanced Metrics in CloudWatch
How the metrics are emitted
The Deepgram container writes records in CloudWatch Embedded Metric Format to stdout. SageMaker AI forwards that output to the endpoint’s CloudWatch log group, and CloudWatch Logs automatically extracts the EMF records into standard metrics. This path eliminates the need for a separate collector, sidecar, or additional IAM role beyond the logging permissions already granted to the endpoint.
What the metrics contain
Metrics are emitted under the Deepgram/SageMakerInference namespace and include low‑cardinality dimensions for category, model, and transport. They aggregate across all Deepgram endpoints in a given account and region, deliberately omitting endpoint names or instance IDs. The values correspond to the consumed‑unit counts used by AWS Marketplace metered billing, enabling a direct reconciliation of the AWS bill against actual traffic. No personally identifiable information—such as transcripts or request IDs—is included.
Prometheus and OpenTelemetry Export
In addition to billing metrics, Deepgram now surfaces engine‑level Prometheus metrics and per‑GPU/host statistics. SageMaker AI’s detailed observability feature scrapes these metrics directly from the container, making them queryable with PromQL from CloudWatch, Grafana, or any compatible Prometheus client. The data set includes GPU utilization, host resource usage, and model‑specific counters, providing the granularity needed for performance tuning and capacity planning.
Operational and Security Considerations
- No outbound network required. The container remains isolated per AWS Marketplace constraints, preserving the security posture of network‑isolated deployments.
- Minimal IAM footprint. Only the existing logging permissions are needed; no extra policies are introduced.
- Aggregation limits. Because Deepgram Enhanced Metrics aggregate across endpoints, per‑endpoint troubleshooting must rely on the Prometheus/OpenTelemetry stream.
- Metric cardinality. Low‑cardinality dimensions keep metric volume manageable, but practitioners should monitor the total number of series when scaling to many models or transports.
- Compliance alignment. All data stays within the customer’s AWS account, supporting residency and audit requirements without exposing raw audio or transcripts.
Related CloudNinjas coverage: AWS.
What This Means For Practitioners
To take advantage of the new observability, enable the Deepgram container’s EMF output and confirm that SageMaker AI is forwarding logs to CloudWatch. Add the relevant metric names to existing dashboards or alarm definitions to track usage‑driven cost drivers. Activate SageMaker’s detailed observability to collect the Prometheus and OpenTelemetry streams, then integrate them into your preferred monitoring stack for per‑GPU analysis. Finally, compare the Deepgram‑specific metrics with SageMaker’s built‑in EnableEnhancedMetrics to decide which view best supports your scaling and cost‑optimization workflows.