The Amazon Great Indian Festival and Flipkart Big Billion Days have started.
Millions of users are visiting these platforms, searching for products, checking offers, adding items to carts, and placing orders—sometimes at almost the same moment.
But behind every search, product page, cart update, and payment, there are thousands of backend requests being processed continuously.
So the interesting question is:
How can an application handle millions of users and huge traffic spikes without bringing the system down?
One possible approach is to build a highly scalable architecture using Amazon EKS (Elastic Kubernetes Service) along with load balancing, Kubernetes autoscaling, caching, databases, and other supporting services.
The important part isn't simply running an application inside EKS.
The real challenge is making the infrastructure react to changing traffic.
When traffic increases, the application needs more capacity.
When traffic decreases, that extra capacity should no longer be required.
This is where Kubernetes autoscaling becomes interesting.
Note: Amazon and Flipkart's actual production architectures are proprietary. This article describes a conceptual architecture for understanding how an EKS-based platform can handle large-scale e-commerce traffic.
The problem: traffic doesn't arrive evenly
On a normal day, an e-commerce application might receive a relatively predictable amount of traffic.
For example:
10,000 requests/minute The infrastructure can be sized around this workload.
But when a major sale begins, the traffic pattern can change very quickly.
It might look conceptually like:
Normal | v 10K requests/min | v 50K | v 200K | v 500K+ The exact traffic numbers are hypothetical, but the engineering problem is real:
Traffic can increase much faster than normal capacity requirements.
Running peak capacity all day would be expensive.
Running too little capacity during the spike could result in:
- High latency
- Failed requests
- Timeouts
- Poor user experience
- Overloaded backend services
So the infrastructure needs to be elastic.
A simplified EKS architecture
A simplified architecture could look something like this:
Users | v Route 53 | v CloudFront | v Application Load Balancer | v EKS Cluster | +-----------+-----------+ | | | v v v Product Cart Search Pods Pods Pods | | | +-----------+-----------+ | +--------+--------+ | | v v Cache Database The idea is simple:
Don't make one server handle everything.
Instead, distribute the workload across multiple application instances.
Step 1: A user sends a request
Imagine someone searches for a smartphone during the sale.
The request can travel through several layers:
User ↓ Route 53 ↓ CloudFront ↓ Application Load Balancer ↓ EKS ↓ Application Pod The load balancer helps distribute incoming requests across healthy application targets.
Instead of sending every request to one application instance, traffic can be spread across multiple Pods.
For example:
Load Balancer | +---------+---------+ | | | v v v Pod 1 Pod 2 Pod 3 If traffic increases, additional Pods can be added.
Step 2: Kubernetes manages the Pods
Inside EKS, the application runs as containers managed by Kubernetes.
Suppose the application starts with:
Pod 1 Pod 2 Pod 3 Pod 4 Pod 5 This might be enough during normal traffic.
But then a flash sale begins.
More users arrive.
Instead of manually creating Pods one by one, Kubernetes can use autoscaling policies to adjust the number of replicas.
Before sale: 5 Pods During higher traffic:
5 Pods ↓ 10 Pods ↓ 20 Pods ↓ 40 Pods The actual number depends on the application's resource requirements and scaling configuration.
Step 3: Horizontal Pod Autoscaler
This is where Horizontal Pod Autoscaler (HPA) comes into the picture.
HPA can monitor metrics such as CPU and memory utilization. It can also be configured to use custom or application-level metrics.
For example:
Minimum replicas: 5 Maximum replicas: 100 Target CPU: 60% During normal traffic:
5 Pods As the workload increases:
CPU utilization increases ↓ HPA evaluates metrics ↓ Desired replica count increases ↓ New Pods are created The important concept here is horizontal scaling.
Instead of making one server bigger:
Small Server ↓ Huge Server we can run more application instances:
Pod Pod Pod Pod Pod Pod This is especially useful for stateless services that can handle requests independently.
But what if there isn't enough compute?
There is an important limitation.
Suppose HPA decides that the application needs 50 more Pods.
But the existing EKS nodes are already full.
Node 1 → Full Node 2 → Full Node 3 → Full Kubernetes knows that additional Pods are needed, but there isn't enough capacity to schedule them.
This is where node autoscaling becomes important.
Step 4: Node autoscaling with Karpenter
An EKS cluster can use Karpenter to help provision compute capacity when Pods cannot be scheduled because the existing nodes don't have enough resources.
The flow can look like:
Traffic increases ↓ HPA requests more Pods ↓ Pods cannot be scheduled ↓ Insufficient node capacity ↓ Karpenter provisions suitable compute ↓ New node becomes available ↓ Pending Pods are scheduled Now we have two different scaling levels:
Application level ↓ HPA ↓ More Pods Infrastructure level ↓ Karpenter ↓ More compute capacity This is an important distinction.
HPA scales the application.
Node autoscaling provides the infrastructure needed to run that application.
Step 5: What happens during a traffic spike?
Let's put the pieces together.
Imagine traffic suddenly increases:
Massive Traffic | v Application Load Balancer | +------------+------------+ | | | v v v Pod 1 Pod 2 Pod 3 | | | +------------+------------+ | v HPA | More Pods? | Yes | v Node capacity? | +-----+-----+ | | Yes No | | | v | Karpenter | | | v | More compute | | +-----------+ | v More Pods The result is an architecture that can adapt to changing demand.
Step 6: Why caching is important
There is another problem.
Even if EKS can run hundreds of Pods, what happens if all those Pods send requests directly to the database?
You could end up with:
100 Pods | v Database | X Bottleneck This is why caching can be extremely important.
Frequently requested information can potentially be served from a cache instead of repeatedly querying the database.
For example:
User ↓ Application ↓ Cache ↓ Response Services such as Amazon ElastiCache can be used for distributed caching.
For an e-commerce application, caching can be useful for data such as product information, sessions, or other information where the application's consistency requirements allow it.
The key idea is:
Scaling the application layer doesn't automatically mean the database can handle unlimited traffic.
Every layer needs to be considered.
Step 7: Different services can scale independently
A large e-commerce application usually isn't one giant application.
It may contain different services:
EKS | +----------+----------+ | | | v v v Product Cart Search Service Service Service | | | v v v Cache Database Search During a sale, search traffic might increase dramatically.
Cart traffic might behave differently.
Payment traffic might have another pattern.
Therefore, each service can have its own resource requirements and scaling policies.
For example:
Product Service 10 → 50 Pods Search Service 20 → 100 Pods Cart Service 5 → 30 Pods These numbers are only examples.
The important idea is that not every service needs to scale by the same amount.
Step 8: Scaling down after the sale
This is the part people often forget.
Autoscaling isn't only about adding capacity.
It's also about removing capacity when it is no longer needed.
Suppose the sale ends.
Traffic starts dropping:
500K requests/min ↓ 200K ↓ 50K ↓ 10K HPA can reduce the number of application replicas according to its configuration.
For example:
100 Pods ↓ 70 Pods ↓ 40 Pods ↓ 15 Pods Once there is unused node capacity, node autoscaling can also reduce infrastructure where appropriate.
So the system can move from:
Peak: 30 Nodes 100 Pods toward something closer to:
Normal: 5 Nodes 15 Pods The exact behavior depends on the autoscaling configuration, workload constraints, Pod disruption rules, and other factors.
Autoscaling isn't instant
There is one important thing to understand.
Autoscaling is not magic.
If traffic suddenly changes from:
10K → 500K requests/minute the infrastructure doesn't instantly create everything required.
There is a reaction time.
Metrics need to be collected.
HPA needs to evaluate them.
New Pods need to start.
Container images may need to be pulled.
If additional nodes are required, those nodes need to become available.
Because of this, predictable large-scale events often require capacity planning and pre-scaling, rather than relying entirely on reactive autoscaling.
Pre-scaling before a major sale
For a known event, engineers can increase the baseline capacity before traffic arrives.
For example:
Normal day 5 Pods 3 Nodes Before the sale:
30 Pods 10 Nodes During the peak:
100 Pods 30 Nodes Autoscaling can then handle additional fluctuations.
This is a much safer approach for predictable traffic spikes because the system doesn't have to build all of its capacity after the traffic has already arrived.
The database is still a critical bottleneck
One of the biggest mistakes when thinking about Kubernetes scaling is:
"If I add more Pods, I can handle unlimited traffic."
Not necessarily.
Imagine:
200 Pods | v One database | X Overloaded The application layer may scale perfectly while the database becomes the bottleneck.
Large-scale systems therefore need to think about:
- Read replicas
- Connection pooling
- Caching
- Database scaling
- Index optimization
- Partitioning
- Asynchronous processing
- Queues
- Rate limiting
- CDN caching
- Backpressure
- Graceful degradation
EKS is an important part of the architecture, but it isn't the entire architecture.
What happens behind a simple "Buy Now"?
When you click Buy Now, the request might trigger multiple backend operations.
Conceptually:
User | v Load Balancer | v Cart Service | +------> Inventory | +------> Pricing | +------> Order Service | +------> Payment | v Order Confirmation Each of these components can have different scaling and reliability requirements.
The user sees:
Order Confirmed ✓ But behind that simple message, multiple distributed services may have communicated with each other.
The bigger lesson
When millions of users arrive during a major shopping sale, the solution isn't simply:
"Add more servers."
The real engineering challenge is:
How do we automatically provide the right amount of capacity at the right time?
Too little capacity:
High latency Timeouts Errors Failed requests Too much capacity:
Unused resources Higher cloud costs The goal is elasticity.
Traffic increases ↓ More Pods ↓ More compute when required ↓ Traffic is distributed ↓ Traffic decreases ↓ Pods scale down ↓ Unused capacity can be removed That's the interesting part of EKS.
It's not just Kubernetes running in AWS.
It's the combination of container orchestration, load balancing, autoscaling, compute capacity, caching, databases, observability, and application architecture working together.
Final architecture
Putting everything together:
USERS | v Route 53 | v CloudFront | v Application Load Balancer | v +-------------+ | EKS Cluster | +-------------+ | +-------------+-------------+ | | | v v v Product Search Cart Pods Pods Pods | | | +-------------+-------------+ | HPA | More Pods required? | v Node capacity? / \ Yes No | | | Karpenter | | | v | More compute | | +-----------+ | v Cache / Database The next time you see a massive sale and thousands of products being sold within seconds, remember that there is a huge amount of engineering behind that experience.
Traffic comes in.
Pods scale out.
Compute capacity grows.
Requests are distributed.
Caches reduce database pressure.
And when traffic goes down, the infrastructure can scale back.
That's the real power of designing applications for elasticity instead of fixed capacity.
