Think Clearly

◇ Before you start

One idea carries this course, and it is worth stating before anything else so you can decide whether you accept it.

Judgement cannot be inspected from the inside. Your confidence in a conclusion does not track its accuracy. Your memory of why you decided something is rewritten by knowing how it turned out. Your sense of having considered the alternatives is generated after the fact, cheaply and convincingly. None of this is a personal failing and none of it is fixed by knowing about it. It is how the equipment works.

What can be inspected is a record. A sentence written before the outcome, with a date on it, is a different kind of object from the same sentence recalled afterwards. It cannot be quietly revised. It can be checked. It can be shown to someone else. Everything this course teaches is either a way of producing such a record, or a way of using one.

That commitment has three consequences you should expect.

One: the writing is the method, not a report on the method. When the course asks you to split a situation into fact, inference and unknown, the value is not the tidy page. It is that inference written in its own column stops passing itself off as fact. Do this in your head and you will conclude, accurately, that it changes nothing.

Two: structure sits at the boundary, not in your intentions. A rule you have to remember to apply under pressure is a rule that fails under pressure. The controls in this course are placed where they cannot be skipped: a view written before the meeting rather than a resolve to think independently in it, a resolution date entered when the forecast is made rather than a promise to follow up.

Three: the record is what makes you correctable. A person who cannot be shown to have been wrong cannot improve. Five dated decision records make a pattern visible that twenty years of experience will not, because experience is stored as a story about competence and the record is stored as it was.

Four things get fixed now, before Level 0, and do not change for eight weeks.

The live decision. Name one decision you personally face in the next eight weeks that is genuinely open, matters, and has an owner who is you. It runs through every level: split in Level 0, graded in Level 1, questioned in Level 2, framed and recorded in Level 3.

The claim you have never checked. Name one figure or assertion you repeat professionally and have never traced to its origin. Level 1 traces it. You may keep repeating it afterwards. You may not keep repeating it unexamined.

The room. Name one recurring meeting or forum where decisions are made, at least one participant is senior to you, and you have at some point disagreed silently. Level 4 runs against that room.

The record. Decide now where your decision records will live and in what form, and make the first entry blank and dated today. A record started in week six covers two weeks. A record started today covers the course.

§ Non-negotiable

Seven rules. Where a rule rests on published evidence, it is marked. Where it is a design choice, it is marked as that, because a course about examining claims should not slip its own preferences past you unlabelled.

One. Everything runs against a live situation. No invented scenarios, no tidy vignettes with all the information present. Design choice. A case study has no politics, no missing data and nobody who disagrees, which is why people who are excellent at case studies are frequently poor practitioners.

Two. Everything is written and dated. A judgement you did not write down is not available for review, because what you will recall is the version compatible with the outcome. Evidenced. Telling people an outcome shifted their stated prior probability of that outcome in all 24 test cases in the founding study, by an average of 10.8 percentage points (Fischhoff, 1975), and the effect has since been confirmed across more than a hundred studies.

Three. Every claim carries where it came from. Not a footnote. In the sentence: what you saw, what you were told and by whom, what you inferred. Design choice. Provenance is cheap to record at the moment of writing and expensive to reconstruct later, and the reconstruction is where the errors enter.

Four. Confidence is stated as a number or a fixed word, never as a tone. "I am fairly confident" tells the reader nothing they can use, and tells you nothing you can score. Design choice, with evidence behind it. Across a range of tasks, intervals people offer as 90 per cent confident contain the answer well below 90 per cent of the time, and in some studies below half (Moore and Schatz, 2017). A number is the only version of confidence that can be found to be wrong.

Five. A decision is judged on the record, not on the outcome. Evidenced. Presented with decisions identical in every respect except how they turned out, evaluators rated the good-outcome version higher in 44.3 per cent of paired judgements and lower in 9.3 per cent (Baron and Hershey, 1988). If you judge your own decisions by results, you will learn from noise.

Six. Silence is a fault, not a status. A question nobody asked, a check nobody ran, a dissent nobody voiced: each is recorded as a gap, never treated as agreement. Design choice. The characteristic failure of a review process is that absence of objection renders as absence of problems.

Seven. You remain the one who answers for it. No source, no group, no seniority and no machine takes on the accountability for a judgement you signed. Design choice, and the position of every framework examined for this course. Where a tool produced part of the reasoning, the tool is an input to your judgement, and the sentence you say out loud is that you reviewed it and you stand behind it.

◈ Reference

Six components. The weights are a design choice; what each demands is not.

Situation work, 10 per cent. Splits of real situations into established fact, inference and unknown, with sources named. Marked on whether the inferences are genuinely separated, not on how few of them there turned out to be. A split in which everything is a fact has not been done.

Information discipline, 20 per cent. One claim traced to its origin or shown to have none, five habitual sources graded and defended, and one questions-only session run against a real meeting, with the fact it surfaced that was not going to be volunteered. Marked on the trace, including the dead ends, on whether the grading survives an argument with someone who ranks them differently, and on the question record rather than on how the meeting went.

Decision work, 20 per cent. A decision frame on the live decision, ten forecasts recorded with numeric confidences and resolution dates, and one decision made on the record with the essential uncertainty still open. Marked on whether the confidences were stated before resolution and whether the reversal condition was written and specific.

The room, 15 per cent. A pre-committed view written before a real meeting, and one occasion of stating a position against the room's direction, with what you said and what happened. Marked on the wording and the record, not on whether you won.

Machine work, 10 per cent. One verification record on a generated artefact you actually used for a real purpose: the claims listed before reading, each marked checked, unverifiable or rejected, and the not-checked list stated with reasons. Marked on the record, including the checks that came back clean, and on whether the not-checked column is honest rather than empty.

The thirty day decision lab, 25 per cent. Five completed decision records, one detailed decision analysis judged on information available at the time, a scored forecast set, a written pattern analysis, and three named changes to how you decide. Marked on the record. A lab with five successes and no corrections has been curated.

Nothing is assessed on the outcomes of your decisions.

† Evidence

Every load-bearing figure in this course carries a citation and one of three grades.

Evidenced means a meta-analysis, several independent studies, or one large study whose full text was read.

One study means exactly that, with the sample stated so you can weigh it.

Convention means no adequate test exists. This course says so at the point of use rather than teaching silence.

The structural fact about this field. In judgement research the problems are measured far better than the remedies. Anchoring came back from a 36-sample replication larger than the original. Sunk cost, hindsight bias, outcome bias, conformity and the bias blind spot all replicate. Now look at the other column. The best evidence for bias training is two uncontrolled pre-post studies and one non-randomised quasi-experiment from a single research group. The surgical safety checklist that reduced mortality in eight volunteer hospitals produced a null result across 215,000 procedures in 101 hospitals under mandated rollout. Nudging as a class went from a pooled effect of 0.43 to 0.04 after correction for publication bias, and the two largest real nudge units average 1.4 percentage points against 8.7 in journals. Decision journals, the single most promoted judgement practice in professional circles, have no controlled evidence at all.

This course is designed against that asymmetry rather than around it. Where the remedy is untested, it says so and gives you a reason to use it anyway or a reason not to. Where the remedy is tested, the figure is here.

Claims you will not find in this course, and why.

"Learning about biases helps you avoid them." Removed. The bias blind spot replicates at d = 1.72 in a preregistered replication, is stable on retest at r = .80, is uncorrelated with measured decision-making competence, and moderates debiasing training so that the people most sure they are unbiased gain least from it.

"Experts are protected by expertise." Qualified rather than removed. Legal professionals anchored on a randomly generated number across four studies. Radiologists at every experience level lost accuracy when an automated suggestion was wrong. But intuition is genuinely trustworthy where the environment is regular and feedback is rapid and unequivocal, and Level 3 teaches the test rather than a slogan.

"Groupthink explains bad group decisions." Removed as an explanation. Four decades of review find the model has almost never been tested whole and that cohesion, its signature antecedent, repeatedly fails to predict anything. Level 4 teaches the parts that are evidenced: conformity, biased information sampling, and hierarchy.

"Under pressure, judgement collapses." Removed. The meta-analysis of stress and decisions under uncertainty puts the pooled effect at d = 0.17, and at 0.01 where reward is not salient. Pressure does something narrower and more useful to know, and Level 4 states what.

"Premortems improve decisions by 30 per cent." Removed. The 1989 study behind that figure counted the number of reasons people generated. It never assessed whether the reasons were correct, and the word "correctly" was added by a later practitioner article. The technique is still taught here, with its actual standing stated.

"The Yerkes-Dodson law shows an optimal level of stress." Removed. It is a 1908 experiment on fewer than forty dancing mice learning a visual discrimination under electric shock, and the inverted-U curve now attributed to it does not appear in the paper.

Where the evidence does not reach. No study measures whether keeping a decision record improves subsequent judgement. No study isolates prediction-scoring from training and practice. No controlled trial of teaching source evaluation includes a durability follow-up. There is no direct experimental test of whether a senior person stating a view first changes the judgements that follow, only the adjacent finding that a first-stated number from any source moves expert estimates. Each of these is marked where it arises, and the exercise that stands in its place asks you to take the measurement on yourself.

‡ Worked to standard

Three working artefacts, reproduced in their real format and annotated. They are the formats this course teaches, shown at full strength rather than as blank templates.

The situation split. One real workplace situation, divided into established fact with sources, inference, and unknown. Annotated at every point where a sentence that reads as a fact turns out to carry an inference, and at the two places where naming the unknown changed what the reader would do next.

The decision record, before and after. The same decision written the week it was taken and reviewed four months later. Annotated for what the original entry got right, what the review could only see because the entry existed, and the one line the author would have sworn was in the original and was not.

The verification record. A machine-generated briefing marked up claim by claim: checked, unverifiable, rejected. Annotated for how long each check took and for the one confident sentence that survived three readings before anyone tried to find its source.