A scheduled live event is a capacity problem with the guesswork removed. Most of the time you’re speculating: maybe you’ll get popular, maybe a link goes big. Not here. On a known date, at a known time, a known-ish number of people all arrive at once, and the thing either holds or it doesn’t, in public.
The good news is that a known spike is the easiest kind to prepare for, because you can rehearse it. The bad news is that most load testing produces a comforting number that means nothing, because the test didn’t resemble reality and nobody checked what actually broke. Here’s how I run one so the number is worth trusting.
Test the shape of real traffic, not a round number
“Can it handle a thousand users” is the wrong question, because a thousand users doing different things stress the system in completely different ways. The right question is: what does a real session look like, and what happens when a realistic mix of them arrives on the real ramp?
For an event, the ramp matters as much as the peak. People don’t trickle in; they pile in when the thing starts. So the test isn’t a flat thousand users. It’s a ramp from near-zero to peak over a few minutes, held at peak, then a tail. If your system autoscales, the ramp is precisely what you’re testing: can it add capacity faster than the crowd arrives? A flat test never asks that question and so never finds the answer.
// k6: model the arrival curve, not a flat load
export const options = {
stages: [
{ duration: '3m', target: 800 }, // the rush when it kicks off
{ duration: '10m', target: 800 }, // hold at peak
{ duration: '3m', target: 0 }, // tail off
],
thresholds: {
http_req_duration: ['p(95)<2000'], // decide "acceptable" before you start
http_req_failed: ['rate<0.01'],
},
};Set the thresholds before the run. “Acceptable” defined after you see the results is not a standard, it’s a rationalisation.
Mock the expensive upstream
The app I was testing sat in front of a paid, rate-limited AI API. Two problems with hammering the real thing during a load test: it costs real money per call, and the upstream’s own rate limits mean you’d be testing their throttle, not your system. So I put a mock in its place: a tiny service that returns a realistic-shaped response after a realistic delay.
# mock upstream: realistic latency, realistic payload, zero dollars
@app.post("/v1/messages")
async def mock():
await asyncio.sleep(random.uniform(0.8, 2.5)) # mimic real think-time
return {"content": LOREM, "usage": {"tokens": 900}}This isolates the variable you actually care about, which is your own infrastructure: your load balancer, your container scaling, your connection handling, your queueing. You are not testing the AI vendor. You are testing whether your stuff falls over on the way to and from the AI vendor. Mocking the upstream also means you can run the test as many times as you need without watching a bill climb, which means you actually run it more than once, which is the whole point.
Watch what breaks, not just whether it broke
A pass/fail number is the least useful output of a load test. The value is in watching the system under stress and seeing where it strains first. During the runs I kept an eye on the things that actually predict an outage:
- Where latency climbs. p95 creeping up before failures start is your early
warning. It tells you the bottleneck’s location before it becomes an outage.
- Whether autoscaling keeps pace with the ramp. New capacity has a cold-start
cost. If the crowd arrives faster than tasks come healthy, you get a few minutes of pain right at the start, which is the worst possible moment. That’s an argument for pre-warming before a known event.
- Errors by type. Timeouts, 5xxs, and connection resets fail for different
reasons. The mix tells you whether you’re out of compute, out of connections, or out of something you hadn’t thought about.
Pre-warm for a known spike; don’t be a purist
Autoscaling is wonderful for unpredictable traffic. For a scheduled event, being a purist about it is just choosing to eat the cold-start pain in public. If you know the crowd arrives at a specific time, raise the minimum task count before it and let it scale down after. It costs a little extra for a couple of hours and removes the single riskiest window from the day. Boring, cheap, effective.
The boring conclusion
A load test is only worth running if it resembles the day you’re preparing for and if you watch what strains. Model the real arrival ramp, not a flat number. Mock the expensive upstream so you’re testing your own system and not someone else’s throttle, and so the test is cheap enough to repeat. Decide what “acceptable” means before you start. Watch latency and scaling behaviour, not just the pass/fail. And for a spike you can see coming, pre-warm and stop being clever. The reward is that when the event arrives, the interesting thing happening is the event, not your infrastructure.
Leave a Reply