Why moderation is a pricing problem, not a values problem

A platform announces a commitment to removing harmful content. It has two billion posts a day and a moderation budget that divides to a fraction of a penny per post. The policy is sincere; the arithmetic is the constraint.

What everyone sees

Moderation failures are attributed to values: the platform does not care enough, or it has the wrong priorities. The alternative reading — that the platform’s values are constrained by a cost-per-decision that no amount of caring can overcome — is less satisfying, because it removes the blame that makes the story actionable.

What is actually happening

Gillespie described the scale problem of content moderation: at billions of items, even highly accurate automated systems produce a volume of errors that exceeds the capacity of human review. Roberts documented the economics of commercial content moderation, showing that human moderators are hired at low cost, in high volume, in jurisdictions where labour is cheap. The result is that the quality of moderation is a function of the cost the platform is willing to spend per decision, which is determined by the revenue each piece of content generates, not by the platform’s stated values.

Why it stays hidden

The economics hide behind the values announcement. The platform publishes principles, which are free to write, while the constraint operates in the cost-per-review, which is not published. A user who sees harmful content attributes the failure to a moral choice, because the values statement implies that the platform has chosen what to do and is failing to do it. The actual failure is arithmetic: the cost of catching everything exceeds the revenue of showing everything.