Scaling Trust and Safety Decision Systems Across Billions of Content Items

Abstract

A large-scale music streaming platform operating a catalogue of more than 250 million tracks, with tens of thousands of new tracks arriving daily through distributor pipelines, cannot review its content by hand. At one minute per track, a single pass over the catalogue would consume centuries of person-time. Machines therefore make the decisions, and human judgement is reserved for a narrow uncertain band. This paper is a practitioner account of the machine decision system that resulted. It argues that the hard problems at this volume are not classification problems but governance problems: where a threshold sits, who owns it, which decisions may be automated, what counts as evidence, and what happens when the institution has not decided what a category means. Six design commitments carried the system. Harm is treated asymmetrically: precision is a promise to individuals, recall a promise to the ecosystem, and the false-positive bound is expressed as a count of artists wrongly flagged per month rather than as a rate. Review capacity is allocated on two axes, model uncertainty and stakes, and review is conducted per campaign rather than per item. The historical backfill is separated from the live path by a hard capacity reservation and sequenced by economic urgency rather than upload date. Detection, classification, and enforcement are kept as separate layers, so that the model measures and the policy decides, and correctness is estimated through converging evidence rather than a single oracle. The account is retrospective, no controlled evaluation is reported, and no operational metric is released; each place where a number would ordinarily stand is marked.