The reason a required field decides what data exists

A form marks industry as required and company size as optional. A year later the database has complete industry data and company size for a third of records. Analysis of that third is presented as a picture of the customer base. It is a picture of the customers who chose to answer.

What everyone sees

Missing data is treated as an inconvenience to be handled statistically — imputed, weighted or excluded. What is less often examined is whether the missingness is informative, and in an optional field it almost always is, because the decision to answer is made by the respondent according to their own situation.

What is actually happening

Rubin’s framework distinguishes data missing completely at random from data missing not at random; optional fields typically produce the latter, since willingness to disclose correlates with what there is to disclose. Heckman’s work on selection showed that estimates from self-selected samples are biased in the direction of the selection mechanism, not merely less precise. Groves documented that nonresponse bias is driven by the correlation between response propensity and the measured variable. Requiring a field does not just raise completeness; it removes the respondent’s ability to sort themselves out of the record.

Why it stays hidden

The bias hides because the collected records look clean. Every value present is accurate, no error is visible, and the sample is often large enough to feel authoritative. Absence leaves no artefact in the table, so the analyst sees a smaller dataset rather than a skewed one, and small is a familiar problem with familiar remedies while skewed is neither.