ProblemsML0026

Undeployable Despite Good Offline Metrics

Medium

Problem

A team ships a 2-point accuracy win and gets paged at 2am: the new model is roughly 3x too large for the serving fleet, and p99 latency tripled the moment it went live. The release process had checked exactly one thing; accuracy; and nothing else. Nobody's buying new hardware to fit this model; the serving fleet is what it is, handling roughly 500 requests/second at a p99 budget of 150ms, same as before the release.

Required components

Click to add a component to the canvas.

Additional components

Browse by category. Click or drag a component onto the canvas.

Drag components from the library onto the canvas, then drag from a component's right handle to connect it to another.

100%

Design canvas

Sign up to start designing

Sign up free

Premium is free for 58 more days.