Making the Slow Thing Fast

A conference talk · The Midnight theme

Measure first

p99 latency, 10k requests, warm cache
Changep50p99RPS
Baseline 18 ms 240 ms 4,100
Pooled connections 14 ms 121 ms 5,900
Batched writes 11 ms 58 ms 8,400
Both9 ms 31 ms 11,200

The hot path, before

for (const id of ids) {
  // One round trip per id. This is the bug.
  const row = await db.get('users', id);
  results.push(shape(row));
}

The hot path, after

const ids = req.query.ids.split(',');
const rows = await db.getMany('users', ids);
const results = rows.map(shape);

One call. N round trips become 1.

Part two: keeping it fast

Regression tests, not vibes

Guard the win

Assert the budget

assert.ok(
  p99 < 50,
  `p99 was ${p99}ms`
);

Why a budget beats a baseline

  • Survives new hardware
  • Fails loudly, not statistically
  • Reads as a product decision
Premature optimization is the root of all evil.
— Donald Knuth, Structured Programming with go to Statements, 1974

Measure. Fix one thing. Measure again.

Thank you

Questions · Slides built with Dais