A system serving around 500 concurrent users on a normal day, jumping to 8,000–10,000 within minutes at peak. Buying hardware for the peak wastes money all year; keeping the everyday configuration fails at the moment that matters most.
This is a reference architecture, not a client case study. No client names or figures from a specific project appear on this page.
The problem
A system serving around 500 concurrent users on a normal day. During peaks — a sale, registration opening, a filing deadline — that jumps to 8,000 or 10,000 within minutes, holds for a few hours, then returns to normal.
Buying hardware for the peak means carrying idle infrastructure 95% of the year. Keeping the everyday configuration means the system fails exactly when it matters most, which is usually when revenue is concentrated.
Proposed architecture
Separate tiers so they scale independently
Three separate tiers: static content delivery, application logic, and the database. Each scales on its own, because each comes under a different kind of pressure.
Push as much as possible to the edge
Images, CSS, JavaScript, and even pages whose content changes rarely, served from a content delivery network. Most peak traffic then never reaches the application servers. This gives the highest return for the effort spent.
Stateless application tier
Application servers hold no session state, so instances can be added and removed freely. Set the scaling trigger on response latency rather than CPU utilisation — latency is what users actually feel.
Split database reads and writes
This is the hardest constraint, because databases do not replicate as easily as application servers. The approach: one primary accepting writes, several read replicas sharing query load, plus a cache layer for repeated queries.
Queues for anything that can wait
Confirmation emails, report generation, syncing to other systems — push them onto a queue. During a peak, users get an immediate response while the heavy work runs behind.
Failure points and how to handle them
| Weak point | Mitigation |
|---|---|
| Scaling cannot keep up with the ramp | Pre-scale on schedule when peak timing is known; keep warm capacity in reserve |
| Cold cache after a restart | Pre-load hot data before the peak window |
| Database connection exhaustion | Connection pooling with per-service limits |
| One slow service drags everything down | Timeouts and circuit breakers |
| Cloud bill exceeds budget | Spending caps and daily budget alerts |
Delivery sequence
- Measure first. Do not optimise before knowing where the bottleneck is. The result often surprises people — it is usually one database query rather than server CPU.
- Fix the cheap things. Add indexes for slow queries, enable compression, move static assets to a CDN. Many systems need nothing beyond this step.
- Remove state from the application tier. A prerequisite for scaling out.
- Build autoscaling and load test it. Test at 1.5 times the expected peak.
- Rehearse before the real peak. Run the full scenario, including the failure cases.
Commonly overlooked
Load testing must simulate real behaviour, not hammer one endpoint. Real users log in, search, view details, add to basket and check out — each step stresses a different part of the system. A badly designed load test produces reassuring numbers and the system still falls over on the day.
What you would need to provide
- At least six months of real traffic data, if available
- Expected peak magnitude and when it occurs
- Your threshold: how slow counts as broken
- A ceiling for infrastructure spend during peaks
Written from the team's hands-on experience delivering infrastructure and software. No sponsorship, no product placement.