The catalogue that does not work

The false picture to remove here is the one most likely to have brought you to a course like this: that if you learn what the biases are called, you will notice them operating and correct for them.

The evidence against that is unusually direct, because it was measured on the belief itself. People asked to rate how susceptible they are to a list of biases, compared with the average person, rate themselves as less susceptible. The effect is large, it replicates in preregistered work at d = 1.72, and it is stable on retest at r = .80 over eight days, which means it behaves like a personality trait rather than a mood. Critically, participants held their self-assessments after being given a full explanation of the bias in question (Pronin, Lin and Ross, 2002; preregistered replication, N = 203, 2024). Being told what the bias is and how it works did not change the belief that it applies to other people.

Two further findings finish the argument. Bias blind spot scores are uncorrelated with performance on standard decision-making competence batteries, all correlations under 0.14 (Scopelliti et al., 2015; total N = 1,527). And in that same programme, debiasing training reduced a target bias only among participants whose blind spot scores were below a threshold, and not among those above it. The people most confident they are unbiased gained least from the intervention designed to help them.

Now the intervention literature, because you should know what the strongest available evidence looks like before deciding what to do with it. The best-supported debiasing training is a set of interactive games. Across two studies, participants improved on bias tests by 32 per cent immediately and 24 per cent at follow-up of eight and twelve weeks, with effect sizes above d = 1.1 (Morewedge et al., 2015; N = 243 and 238 analysed). One field extension found trained business students chose the hypothesis-confirming option in a classroom case 58.8 per cent of the time against 72.2 per cent untrained (Sellier, Scopelliti and Morewedge, 2019; N = 290).

Read those studies properly. Neither of the 2015 experiments had a no-training control group, so every reported effect is a within-subject change in a trained group with practice effects uncontrolled. The outcomes are scores on bias tests, not decisions. The field study was not randomised, and the outcome was a single binary choice on a classroom case at a mean of eighteen days. All three come from one research group and no independent replication exists. This is the best evidence there is, and it is not strong evidence.

That is not a reason to do nothing. It is a reason to be precise about what you are buying and where you place it.

The wider context makes the caution worth carrying. Behavioural interventions as a class have a publication problem large enough to change the sign of conclusions. A meta-analysis of 447 effect sizes across 2.1 million people put the pooled effect of choice-architecture interventions at d = 0.43. A re-analysis correcting for publication bias put it at d = 0.04, with a confidence interval touching zero (Mertens et al., 2022; Maier et al., 2022). Independently, a study covering every trial run by two of the largest applied units, 126 randomised trials across 23 million people, found an average effect of 1.4 percentage points against 8.7 in the published academic literature, attributing roughly 70 per cent of the gap to selective publication and low power (DellaVigna and Linos, 2022).

The same shape appears in the most celebrated structural intervention of the last twenty years. A surgical safety checklist cut deaths from 1.5 to 0.8 per cent and complications from 11.0 to 7.0 per cent across eight volunteer hospitals with intensive implementation support (Haynes et al., 2009). When checklists were mandated across 101 hospitals in Ontario and measured over roughly 215,000 procedures, mortality moved from 0.71 to 0.65 per cent, odds ratio 0.91, p = 0.13, and complications did not move at all (Urbach et al., 2014). The intervention did not fail because checklists are foolish. It failed because a control that works when a team has chosen it and been supported through it is a different object from the same control imposed on people who are getting through it.

So what does this course do instead? Three things, and they are the design commitments the rest of the levels implement.

It puts the control where it cannot be skipped rather than where it must be remembered. A view written before the meeting beats an intention to think independently in it. A resolution date entered when the forecast is made beats a plan to follow up. The evidence for this preference is indirect but consistent: the one intervention with controlled evidence against overreliance on machine output was structural, withholding the answer until the person had committed to their own (Buçinca et al., 2021).

It targets the four patterns that survived large-scale replication, not the catalogue. That is the next lesson.

It relies on the record rather than on the reasoner. Everything the course asks you to write is designed to be checkable later by someone who was not there, including you.