ProblemsML0025

Aggregate Accuracy Hides a Segment Regression

Medium

Problem

A new model beats the old one on aggregate accuracy and ships. Two weeks later, support tickets spike: it's failing badly for users in one region on one device tier, a regression the aggregate number never had a chance of catching. Rollback takes 6 hours because nobody built one. Design the model promotion pipeline so a segment-level regression is caught before full rollout, and a bad one can be pulled back in minutes, not hours.

Required components

Click to add a component to the canvas.

Additional components

Browse by category. Click or drag a component onto the canvas.

Drag components from the library onto the canvas, then drag from a component's right handle to connect it to another.

100%

Design canvas

Sign up to start designing

Sign up free

Premium is free for 58 more days.