ProblemsML0007

Human Labeling Throughput

Medium

Problem

A backlog of 50,000 unlabeled examples is sitting behind the training pipeline, and new items keep arriving faster than the labeling team can clear them. Size a labeling queue and its connected labeler workforce to actually drain the backlog in a reasonable window; and since a single reviewer's label is noisy, route items through a consensus review mode for quality, even though consensus costs real throughput. A compliance ask on top: someone has to spot-check a slice of what ships, after the fact, independent of the people doing the original labeling.

Required components

Click to add a component to the canvas.

Additional components

Browse by category. Click or drag a component onto the canvas.

Drag components from the library onto the canvas, then drag from a component's right handle to connect it to another.

100%

Design canvas

Sign up to start designing

Sign up free

Premium is free for 58 more days.

$0 / $600