ProblemsEX0021

Logging Platform

Hard

Problem

During last month's traffic spike, the logging pipeline silently dropped roughly 30% of incoming logs, exactly the window every on-call engineer needed them most, and nobody noticed until the postmortem. Separately, one noisy service's debug-level logging once ate enough shared ingest capacity that other teams' logs started lagging behind. Build the centralized logging platform every service ships its logs to, with a search API engineers use to debug production issues. Design it from a blank canvas so a burst in log volume is absorbed and never dropped, one tenant's volume can never crowd out every other tenant's logs, a search for something that happened five minutes ago always comes back fast from a real index rather than a raw scan, a record that fails to parse is set aside rather than silently dropped or blocking everything behind it, and logs old enough that nobody's actively debugging against them move somewhere cheaper automatically.

Required components

Click to add a component to the canvas.

Additional components

Browse by category. Click or drag a component onto the canvas.

Drag components from the library onto the canvas, then drag from a component's right handle to connect it to another.

100%

Design canvas

Sign up to start designing

Sign up free

Premium is free for 58 more days.

$0 / $2,600