How United Airlines solved IP exhaustion with Private NAT Gateway

United Airlines, a major U.S. airline headquartered in Chicago, Illinois, moves over 181 million passengers annually across a global route network. When severe weather grounds flights or air traffic control issues cascade across the network, United Airlines needs every system to scale instantly, rebooking passengers, reassigning crews, reprocessing flight data. But with hundreds of AWS accounts connected to on-premises data centers through a hybrid network, routable IPv4 address scarcity created a limit on that scale. Every container, AWS Glue job, and AWS Lambda function consumes an IP from a limited pool, and during these surges, the pool is exhausted.

“Every ENI attached to a container, every AWS Glue job, every AWS Lambda function running in Amazon VPC consumes a routable IP address from that limited pool. You reach the ceiling exactly when you need capacity most.”

– United Airlines Engineering

In this post, you’ll learn how United Airlines used to decouple compute scaling from IP address availability. We cover the architecture pattern, route table configuration, and operational lessons from running this in production across hundreds of accounts.

United Airlines’ network team allocates RFC 1918 (private IPv4 address space) from a limited pool to each AWS account. This is standard for large enterprises with pre-cloud network infrastructure. The network team shares address ranges across business units and coordinates each allocation to avoid overlap with on-premises and other Amazon Virtual Private Cloud (Amazon VPC) ranges.

Every elastic network interface (ENI) attached to a container task, AWS Glue job, or VPC-attached AWS Lambda function consumes one IP address from that allocated range. Under normal daily operations, the available addresses are sufficient. During irregular operations (IRROPS) such as severe weather events, air traffic control ground stops, or system-wide disruptions, they are not.

When flight disruptions occur, multiple workload types scale simultaneously:

  • Amazon Elastic Container Service (Amazon ECS) tasks spin up for passenger rebooking, bag retagging, and crew reassignment
  • AWS Glue jobs reprocess flight data, recalculate connections, and update downstream systems
  • AWS Lambda functions fan out across event-driven workflows handling thousands of passenger itinerary changes

All these workloads attach ENIs to subnets and consume routable IPs from the limited pool. Workloads reach the IP ceiling at the exact moment capacity is most needed. Requesting additional routable ranges from the network team requires weeks of coordination, including on-premises firewall rule changes. United Airlines needed a solution that can deliver capacity in minutes.

Figure 1: Workloads on routable subnets consuming limited IP pool during IRROPSFigure 1: Workloads on routable subnets consuming limited IP pool during IRROPS

Before selecting Private NAT Gateway, United Airlines evaluated several approaches:

Approach Assessment
Request additional RFC 1918 space Multi-week lead time. Each allocation requires coordination with the network team and firewall rule changes on-premises. Does not scale with burst demand.
IPv6 adoption Best long-term answer for IP exhaustion. Amazon ECS, AWS Lambda, and Amazon Elastic Kubernetes Service (Amazon EKS) support IPv6, and NAT64 handles translation to IPv4-only destinations. United Airlines’ existing infrastructure (AWS routing, monitoring, security policies) is built on IPv4 across hundreds of accounts. However, adopting IPv6 is an organization-wide decision at United Airlines. Private NAT Gateway solved the IP exhaustion problem within weeks using the existing infrastructure.
AWS PrivateLink Suited for exposing specific services across VPCs. United Airlines workloads require general outbound connectivity to many destinations (Transit Gateway, on-premises, shared services), which PrivateLink does not address.
Amazon VPC Lattice Provides service-to-service connectivity across VPCs and accounts, including to on-premises resources through resource gateways. Well suited when traffic patterns are defined service-to-service calls. United Airlines’ workloads require outbound connectivity to multiple VPCs, shared services, and on-premises systems over Transit Gateway. Traffic is controlled through security groups and Transit Gateway route table segmentation. This pattern was easier to implement using NAT translation than by registering each destination as a Lattice resource.
Network renumbering Removes the root cause but represents a multi-year effort for a network of this size, touching every device, firewall rule, VPN tunnel, and routing advertisement across hundreds of accounts and data centers.

United Airlines selected Private NAT Gateway because it provides general outbound connectivity from non-routable subnets, requires no changes on the destination side, and was deployable within weeks.

United Airlines allocates compute subnets from the 100.64.0.0/10 range (Carrier-Grade NAT space, RFC 6598). These addresses are not routable on the corporate network, so teams can allocate them without coordination and without scarcity constraints.

The trade-off: workloads running on 100.64.x.x subnets cannot communicate with anything outside the VPC. They cannot reach other Amazon VPCs through AWS Transit Gateway, on-premises systems, or shared services.

Private NAT Gateway solves this. Deployed in a routable subnet, it translates the non-routable source address to a routable address as traffic exits the VPC. ECS tasks, AWS Glue jobs, and AWS Lambda functions communicate with the rest of the network normally while running on subnets with no IP scarcity constraint.

For Amazon EKS workloads, United Airlines uses custom networking, which allows pods to use non-routable subnets natively while node primary interfaces remain on routable subnets. ECS, AWS Glue, and AWS Lambda do not have this option because they attach ENIs directly to whatever subnet is configured. Private NAT Gateway solves connectivity for these services.

The following diagram shows the high-level architecture with Private NAT Gateway connecting non-routable compute subnets to the routable network:

Figure 2: Private NAT Gateway architecture non-routable subnets, Private NAT Gateway, routable subnets, Transit Gateway

Figure 2: Private NAT Gateway architecture non-routable subnets, Private NAT Gateway, routable subnets, Transit Gateway

To implement this pattern, the following components are required:

  1. Amazon VPC with a secondary CIDR from the 100.64.0.0/10 range attached. This provides the non-routable address space for compute subnets.
  2. Subnets in both ranges: non-routable subnets for compute workloads, routable subnets for Private NAT Gateway, load balancers, and Transit Gateway attachments.
  3. AWS Transit Gateway (or equivalent connectivity) attached to routable subnets for reaching other VPCs and on-premises networks.
  4. Route table configuration as described in the following section.

Two route tables handle the traffic flow:

Non-routable subnet route table

Destination Target Purpose
100.64.0.0/16 local Communication within VPC
10.5.0.0/16 local Communication within VPC
Pl-xxxx(s3) vpce-s3 S3 Access
Pl-yyyy(DynamoDB) vpce-db DynamoDB Access
0.0.0.0/0 Private NAT Gateway Any other traffic

Routable subnet route table

Destination Target Purpose
100.64.0.0/16 local Communication within VPC
10.5.0.0/16 local Communication within VPC
0.0.0.0/0 Transit Gateway Any other traffic

Private NAT Gateway translates the source IP from 100.64.x.x to its own routable IP address before the packet reaches Transit Gateway. Return traffic arrives at the gateway’s routable IP, and the gateway translates it back to the original 100.64.x.x address. The workload is unaware of the translation.

Figure 3: Route table and packet flow through Private NAT Gateway

Figure 3: Route table and packet flow through Private NAT Gateway

The following table summarizes the operational impact:

Metric Before Private NAT Gateway After Private NAT Gateway
IRROPS scaling constraint IP address availability No longer IP-constrained
Time to add capacity Weeks (network team coordination + firewall changes) Minutes (scaling on non-routable subnets)
Dependency on network team Per-account allocation requests None for compute scaling
ECS/AWS Glue/AWS Lambda task launch failures during spikes Occurred when IPs exhausted No longer occurs

 

Right-size NAT Gateway capacity across availability zones. During IRROPS spikes, throughput through Private NAT Gateway increases sharply. United Airlines deploys multiple NAT Gateways across availability zones. A single gateway supports up to 100 Gbps and 55,000 simultaneous connections per unique destination. With thousands of short-lived AWS Lambda invocations targeting the same endpoints, port exhaustion can occur before bandwidth becomes a factor. Monitor the ErrorPortAllocation CloudWatch metric and consider associating additional secondary IPv4 addresses to increase the per-destination connection quota.

Amazon EKS custom networking and Private NAT Gateway are complementary. Pods receive non-routable IPs natively through custom networking. ECS, AWS Glue, and AWS Lambda receive connectivity through Private NAT Gateway. Both use the same non-routable subnets; the mechanism for outbound traffic differs.

Cost was justified for the enterprise. A single IRROPS event where rebooking systems fail to scale can affect tens of thousands of passengers within minutes. The cost of Private NAT Gateway across all accounts is small compared to the operational and customer impact of a single scaling failure.

Reserve routable subnets for resources that require direct reachability. Load balancers, NAT Gateways, and interfaces that must be addressable from the corporate network remain on routable subnets. Other compute moves to non-routable space.

NAT translation hides the original source IP. When a downstream service receives traffic from Private NAT Gateway, the source address is the gateway’s routable IP rather than the originating container or function. VPC Flow Logs support a custom format that captures both pre-NAT and post-NAT addresses in the same log record:

  • pkt-srcaddr / pkt-dstaddr: original packet addresses before translation
  • srcaddr / dstaddr: addresses on the wire after NAT translation

Example custom format:

${srcaddr} ${dstaddr} ${pkt-srcaddr} ${pkt-dstaddr} ${srcport} ${dstport} ${protocol} ${bytes} ${start} ${end} ${action} ${log-status}

This provides network-level traceability back to the specific compute resource without requiring application changes.

To implement this pattern or explore related approaches, start with these resources:

This post focuses on a different use case: how a specific enterprise uses Private NAT Gateway to solve burst-scaling constraints during unpredictable operational events, for services (Amazon ECS, AWS Glue, AWS Lambda) that cannot use Amazon EKS custom networking natively.

In this post, we showed how United Airlines used AWS Private NAT Gateway to solve routable IPv4 address exhaustion during flight disruptions. By deploying compute workloads on non-routable 100.64.0.0/10 subnets with Private NAT Gateway providing outbound connectivity, United Airlines removed the IP address ceiling from the scaling path. The solution was deployed in weeks, requires no changes to destination systems, and task launch failures because of IP exhaustion no longer occur during IRROPS events.

As United Airlines’ engineering team puts it:

“Private NAT Gateway solved a problem that would have otherwise taken years of network re-architecture. It took us weeks.”

For organizations facing similar constraints (limited RFC 1918 allocations in hybrid networks, long lead times for address space expansion, and burst-scaling requirements during unpredictable events), Private NAT Gateway provides a path that avoids multi-year network re-architecture programs.

To get started, see the Private NAT Gateway documentation. If you have questions or want to share how you handle IP exhaustion in your environment, leave a comment below.

About the authors

Ankush Goyal

Ankush Goyal

Ankush Goyal is a Senior Technical Account Manager at AWS Enterprise Support, specializing in helping customers in the travel and hospitality industries optimize their cloud infrastructure. With over 20 years of IT experience, he focuses on leveraging AWS networking services to drive operational efficiency and cloud adoption. Ankush is passionate about delivering impactful solutions and enabling clients to streamline their cloud operations.

Rammohan Potham

Rammohan Potham (Guest)

Rammohan Potham is a Principal Architect at United Airlines and a founding member of the airline’s AWS cloud engineering team. He specializes in large-scale networking and IP address management, designing solutions that bring real-time visibility and automation to AWS environments spanning over 1,200 accounts. Outside of work, Rammohan enjoys cooking and reading, and has a deep curiosity for science, particularly evolution and astronomy.

Similar Posts

Leave a Reply