ProblemsEX0019

Distributed Scheduler

Hard

Problem

Every job scheduled for midnight fired within the same ten-second window last night, and the single dispatcher process that was supposed to hand them out pegged its CPU and fell badly behind. Build a distributed job scheduler: clients register jobs with a due time, and the platform reliably dispatches each one at (or shortly after) that moment, at a scale where tens of millions of jobs sit registered at any time and the dispatch tier processes roughly 1,800 new registrations and a comparable dispatch rate per second, system-wide. Design it from a blank canvas so the jobs table scales horizontally under that write volume, finding what's due right now is always a cheap, targeted operation rather than a full-table scan or a single process's bottleneck as the job count grows, a burst of synchronized due times never fires as one synchronized spike, and no single dispatcher becomes a single point of failure.

Required components

Click to add a component to the canvas.

Additional components

Browse by category. Click or drag a component onto the canvas.

Drag components from the library onto the canvas, then drag from a component's right handle to connect it to another.

100%

Design canvas

Sign up to start designing

Sign up free

Premium is free for 58 more days.

$0 / $2,300