Teams managing production Kubernetes clusters are frequently evaluating migration paths as the ecosystem shifts toward the Gateway API. For many organizations, the challenge is not just selecting a Gateway API implementation, but designing a migration strategy that minimizes operational risk during cutover. Most engineering teams know they need to migrate, but they are short on the bandwidth to do a proper evaluation, unsure which Gateway API controller is the right long-term bet, and aware that a rushed cutover will drop production traffic the moment DNS TTLs disagree with reality. This piece is a case study of an Ingress NGINX migration we recently ran for a customer on AWS. It walks through how we picked Envoy Gateway, how we tested the migration on our own infrastructure first, why our first successful cutover was not good enough, and the weighted DNS approach that finally got us a clean zero-downtime result.
Why Migrate from Ingress NGINX to Gateway API
Ingress NGINX controller is one of the most widely deployed controllers in the Kubernetes ecosystem, and plenty of production clusters are running it right now. However, with no security patches, no new features, and an Ingress API frozen in place, Gateway API is the logical next step. It is a more expressive and standardized way to define routing rules that aligns with the broader Kubernetes networking evolution. The decision to move is driven by the need for better observability, improved security posture, and the ability to leverage the latest Kubernetes networking standards without being locked into a deprecated API version.
Designing a Zero-Downtime Migration Strategy
The primary goal of any migration is to ensure that user-facing services remain available throughout the transition. A common pitfall is attempting a direct cutover without a phased approach. In our case study, we initially attempted a direct switch, but this resulted in brief outages when DNS propagation lagged behind the actual traffic redirection. To solve this, we implemented a weighted DNS approach. This method allows traffic to be gradually shifted from the legacy Ingress NGINX controller to the new Envoy Gateway instance. By adjusting the weights in the DNS records, we can control the percentage of traffic flowing to each endpoint. This ensures that if the new controller encounters an issue, traffic can be instantly rerouted back to the stable legacy controller without dropping connections.
Testing and Validation Before Cutover
Before touching production, we validated the migration strategy on our own infrastructure. This step is critical for identifying configuration drifts or compatibility issues that might not be apparent in a controlled environment. We focused on testing the specific routing rules, ensuring that the Envoy Gateway could handle the same traffic patterns as the Ingress NGINX controller. This included testing for edge cases like large payloads, high concurrency, and specific header manipulations. Validating these scenarios ensures that the new controller is production-ready. For professionals preparing for Kubernetes certifications, understanding the nuances of controller validation is essential for passing practical exams.
Operational Considerations and Monitoring
During the migration, monitoring becomes even more critical. We set up dashboards to track latency, error rates, and connection counts for both the legacy and new controllers. This visibility allows the operations team to detect anomalies immediately. If the new Envoy Gateway starts showing increased latency, the weighted DNS approach allows us to reduce its traffic share instantly. This proactive monitoring is a key component of a successful migration. It ensures that the transition is smooth and that the team can respond quickly to any issues that arise.
What This Means For You
Implementing a zero-downtime migration requires careful planning and a phased approach. By using weighted DNS and thorough testing, you can minimize the risk of service disruption. This strategy is applicable to any organization looking to modernize their Kubernetes networking stack. Whether you are migrating from Ingress NGINX to Envoy Gateway or another controller, the principles of gradual traffic shifting and rigorous validation remain the same. Adopting these practices will help your team maintain high availability while upgrading your infrastructure.