The Likert Trap: A Postmortem of the 2021 ONS Trust Survey

Dear Marcus,

You asked why I’ve been grumbling about the ONS's 2021 Trust in Government release. Let me walk you through the autopsy, because it’s a textbook case of treating a latent variable like a bathroom scale.

Start with the instrument. The ONS asked: 'How much do you trust the national government to do what is right?' Response options: a five-point Likert from 'A great deal' to 'Not at all'. No 'Don’t Know'. No 'Neither trust nor distrust'. No shade of ambivalence. The question wording itself smuggles in an assumption—that trust is a unidimensional sentiment you can hold toward a faceless apparatus, not a bundle of expectations about competence, integrity, and predictability. A 2019 cognitive testing report (internal, never published) showed respondents hesitating between 'Some' and 'Not very much' because they trusted civil servants but not ministers. The scale forced them to pick one. That’s not measurement; that’s coercion.

Then the mode. This was a web-first push, with telephone follow-up for non-responders. But the web survey presented the scale horizontally; phone interviewers read it vertically. People click left when they see a continuum; they hear endpoints when spoken. Result: web respondents clustered at 'Some' (the middle) while phone respondents skewed toward 'Not very much' (the penultimate negative). The mode effect alone shifted the national estimate by 4 percentage points. We know this because ONS published a mode-effects table in the technical appendix—buried, as if it were a footnote. It wasn’t a footnote; it was a confession.

Now sampling and representation. The sampling frame was the Address-Based Frame, which excludes people in communal establishments: prisons, hostels, student halls, care homes. That’s about 2% of the adult population—disproportionately young, transient, and reliant on government services. Their trust is systematically lower. But we dropped them. Then the weighting adjusted for age, sex, region, but not for housing tenure or benefit receipt. The very people whose trust is most contingent on service delivery were undercounted and underweighted. The published headline—'Trust in government fell to 36%'—is not wrong. It is incomplete. It measures trust among those who have a stable address, a phone, and enough patience to finish a 15-minute survey. That is not the public; that is a convenience sample dressed in weights.

Interpretation is where the real damage happens. The Treasury seized on that 36% figure to justify austerity cuts in public-sector staffing, arguing that low trust necessitated 'efficiency savings'. But the survey didn’t measure trust in service delivery; it measured diffuse political trust. And the drop from 2019 was driven entirely by respondents under 35—the very group excluded from the frame. The statistic was used as a cudgel, not a diagnostic. No one asked why trust was low among those who were actually sampled. No one disaggregated by service touchpoint. The ONS report itself offered no multivariate analysis; it just presented a bar chart over time. That’s not analysis. That’s charting.

Three lessons, Marcus, because I know you’re designing a new local-government trust module.

First, kill the single Likert. Use a multi-item scale that separates trust in competence (can they deliver?) from trust in integrity (do they share my values?). The OECD’s Trustlab framework does this. Steal it.

Second, build mode invariance into the design. If you mix web and phone, test the scale orientation beforehand. Better yet, use a fully anchored visual analogue scale that works identically on screen and voice. It’s possible—I’ve seen it done in the Dutch Safety Monitor.

Third, oversample the excluded. Budget for a booster of residents in communal establishments and people without fixed addresses. Then weight not just by demographics but by service usage. Trust is not a trait; it’s a transaction. Measure it where the transaction happens.

The 2021 ONS survey isn’t useless. It’s just not a measurement of trust. It’s a measurement of how people with landlines and patience feel about a word. And we’ve been treating that word as policy gold. That’s not research. That’s alchemy.

Best, Enzo