ProblemsML0027

Release Without a Champion Comparison

Medium

Problem

A retrained model passes its offline evaluation and goes straight to 100% of traffic on a Friday afternoon. By Monday, conversion is down 4%, and nobody can say for certain whether the model caused it, because it was never run side-by-side against the model it replaced on real traffic first. Design the release system so every new model is compared against the current champion on live traffic, with a bounded auto-halt and a fast rollback, before it ever gets a full promotion decision.

Required components

Click to add a component to the canvas.

Additional components

Browse by category. Click or drag a component onto the canvas.

Drag components from the library onto the canvas, then drag from a component's right handle to connect it to another.

100%

Design canvas

Sign up to start designing

Sign up free

Premium is free for 58 more days.