The Alibi

The Objection That Was Never About Accuracy

#

Paul Meehl published the finding in 1954 and kept seeing patients for another forty years.

The finding was that when you compare a clinician’s judgment against a simple statistical rule, applied to the same information, predicting the same outcome, the rule wins or ties almost every time. He reviewed about twenty studies. The pattern was uniform enough that he expected the argument to be over within a few years, and he said so, and he was a clinician himself, which is the part people forget. He was not an outsider with a formula. He was a practicing psychotherapist who had gone looking for evidence that his own judgment was worth something and had found the opposite, published it, and then gone back to his office on Monday.

By 1986 he was writing that there was no controversy in social science showing so large and consistent a body of qualitatively diverse studies coming out so uniformly in the same direction. By 1996 the count was 136 studies, and the rule was equal or better in roughly nineteen out of twenty, with the exceptions showing no pattern anyone could find. Along the way Robyn Dawes demonstrated the harder version: a linear model with weights assigned at random, provided the signs point the right way, also beats the clinician. The clinician’s contribution was not merely replaceable. It was, on average, negative.

One more finding sits underneath the rest and is rarely quoted. The comparison holds even when the clinician is given the formula’s output. Told what the rule predicts and permitted to adjust, clinicians adjust, and the adjusted prediction is worse than the rule alone. The combination that everyone assumes is the safe compromise, machine proposes and human disposes, was tested early and did not hold. Whatever the human contributes at that point is not correction. It is variance.

Almost nothing changed. Parole boards kept interviewing. Admissions committees kept reading. Hiring managers kept forming impressions in the first four minutes. Meehl called the gap between the evidence and the practice one of the most striking things in the history of applied psychology, and did not manage to close it in his lifetime.

The Objection That Kept Changing Shape

#

Fields resist. What the record holds is the sequence of reasons.

First the studies were said to use inadequate criteria. What counts as a good prediction of parole success or academic performance is a contested question, and if the criterion is wrong then beating the clinician on it proves nothing. This was a real objection and it was answered, over about a decade, by running the comparison across many different criteria in many different domains and getting the same result.

Then the settings were said to be artificial. Laboratory conditions, restricted information, clinicians working without the material they would have in practice. Field studies followed, with clinicians given everything including the interview, and the result did not move. In several designs giving the clinician more information made the prediction worse, because the additional material was mostly noise and confidence rises with volume whether accuracy does or not.

Then came the exceptional case, and this one is Meehl’s own. He named it the broken leg problem: if you know a man goes to the cinema every Tuesday, an actuarial table will predict he goes this Tuesday, and it will be wrong if you happen to know he broke his leg this morning. There are facts the formula cannot see and the clinician can. The objection is correct in principle, and Meehl answered it in practice by pointing out what happens when clinicians are permitted to override. They override far more often than broken legs occur, and the overrides degrade the result, because a clinician who is allowed to make an exception will find that most cases feel exceptional.

Then the objection became that these particular patients, this particular institution, this particular population, are different from the ones in the studies.

A fifth arrived late and is the most modern-sounding: that the rule is opaque, or unfair, or encodes the biases of the data it was fit to. This one has teeth, and it is also the first objection in the sequence that a clinician could raise without any evidence about clinicians at all. It is an objection to the rule that never has to mention the alternative, which is a useful property in an argument you are losing.

Five objections over five decades, each specific, each answered or partly answered, each replaced by another.

An objection that is real names the evidence that would satisfy it. An alibi produces a new objection when the evidence arrives.

What Was Actually Being Defended

#

The standard reading of this history is that professionals protect their income, and it is not wrong and it is not sufficient, because most of these people were not going to lose their jobs. Parole boards were not going to be disbanded. The formula needed someone to gather the inputs and someone to sit in the room and someone to sign. What was at stake was smaller than employment and much harder to give up.

A clinician’s self-concept is not that they hold a job. It is that they can see something. The whole training, the years of supervision, the accumulated sense of a person walking in and something being wrong before anything has been said: all of that resolves into a claim about perception. I notice what the record does not contain. That claim is what the finding attacks, and it attacks it precisely, and it does not offer anything in exchange.

The threat is not that the professional becomes unemployed. It is that the thing they believe makes them who they are turns out to be a rounding error, present in the data as a slight negative coefficient.

This is why the objection kept moving. A person defending an income can be bought out. A person defending a self cannot, because there is nothing to trade. And it explains a feature of the record that the income theory does not, which is that the resistance was strongest among the best practitioners. The people with the most experience had the most invested in the proposition that experience sees things, and the finding is specifically that it does not, or not in the way anyone believed.

Yagn’s framing of this, which he arrived at from the anthropology rather than from the psychology, is that a profession is a claim about what a certain kind of person can perceive, and that every professional guild in history has organized itself around a perceptual monopoly rather than around a set of tasks. Take the tasks and the guild adapts. Take the monopoly on seeing and there is no guild left to adapt.

Where the Objection Was Right

#

Three things in this record run the other way, and any account that leaves them out is contempt with citations.

The metric can be wrong in a way the professional is tracking and the study is not. The rule beats the clinician at predicting the measured outcome. If the measured outcome is a poor proxy for the thing that matters, then the clinician who scores worse on it may be optimizing for something real that nobody has instrumented. This is not a hypothetical objection. It is the ordinary condition of most professional work, where the outcome that gets recorded is the one that was cheap to record.

The second is the broken leg, which is unanswered rather than merely answered on average. Meehl showed that permitting overrides makes things worse in aggregate. That is a claim about a policy, and it leaves untouched the fact that in a specific case, with a specific fact the model cannot see, the clinician is right and the rule is wrong, and the person in front of them absorbs the difference. An institution optimizing aggregate accuracy and a professional sitting with one person are answering different questions, and only one of them has to look at the answer.

The third is accountability, and it is the one that was never really addressed. A formula cannot be sued, disciplined, or asked why. Adopting it requires somebody to hold the consequence of its errors, and in the fifty years of this argument almost nobody offered to. Institutions asked professionals to surrender the judgment while retaining the liability, which is a bad trade that a reasonable person would refuse, and many of them dressed the refusal in an argument about accuracy because an argument about accuracy is the argument that gets you taken seriously in a professional setting. The alibi and the grievance were both present, and the grievance was legitimate.

The Same Costume, Worn Again

#

Every organization currently declining these systems is making an argument, and some of those arguments are findings and some are alibis, and they are indistinguishable in the room.

The test is not the content of the objection. It is whether the objection specifies its own satisfaction. Ask what evidence would change the answer. A finding produces a number, a condition, a study design, a threshold. An alibi produces a pause and then a different objection, and the second objection will be at least as reasonable as the first, which is why this is difficult and why the people making it are not lying. They believe each objection as they make it. What they are protecting is underneath all of them and is not available for inspection, including to themselves.

There is a second tell and it is cheaper to check. Alibis cluster where the identity is thickest. Watch where the objection is raised loudest and compare it to where the actual exposure is, and the two will not match. The exposure sits in the boring, high-volume, low-status parts of the work. The objection concentrates on the part that everyone in the profession considers the real work, which is the part they are best at, which is the part they became this thing in order to do.

Applying the Test Here

#

The central claim of this corpus is that capability commoditizes and deployment does not, because deployment consists of tacit, local, context-specific knowledge held by particular people in particular places who understand how this hospital or this town or this business actually works.

That claim has the shape of the clinician’s objection almost exactly.

It says there is something a knowledgeable person perceives that the general instrument cannot. It says the general case does not apply here because here is specific. It has been amended when challenged rather than abandoned, more than once. And it is held by three authors, two of whom have professional identities built on the proposition, and one of whom is an instrument with its own reasons to say that the humans remain necessary.

I wonder whether any discipline has ever identified its own alibi from the inside, or whether the recognition only arrives from people who were never in the room and had nothing to lose by looking.

The strongest version of the counter-case was put inside this corpus and not by a critic: that firms overweight hyperlocal knowledge because it supplies a sense of uniqueness, that variation between organizations in the same industry is smaller than anyone inside them will admit, and that people defend local knowledge hardest exactly where it is thinnest. That argument was made against this corpus’s own thesis, by one of the people who wrote the thesis, and it has not been answered.

So the test comes back with an uncomfortable result. The deployment claim does specify evidence that would move it, which is in its favor: a pattern library that transfers across firms in one industry, priced as a product rather than staffed as a service, would be most of the refutation. But nobody in this project has committed to a number, a date, or a condition that would count as the claim failing. Until somebody does, the corpus is in the position of a discipline that has answered every objection to its central proposition and has never named the finding that would end it.

Meehl kept practicing. He saw patients into his eighties, having spent his career demonstrating that his clinical judgment was worth less than a table, and he never resolved the contradiction in print. The closest he came was a remark that a psychologist could hold two things at once: that the actuarial method should decide, and that a person in a room with another person is doing something the decision does not describe.

That may be an evasion. It may also be the only honest position available to somebody who proved the thing and then had to go to work on Monday.

This essay does the work that the first entry of The Dissents (The Half-Arrival) conceded in two sentences, where the Meehl finding was granted as evidence against the corpus’s own position and then set aside. It connects to Part 105 (Not Yet), which handles the case where the refusal is rational rather than defensive.

References

#

The Actuarial Finding:

Meehl, Paul E. Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. University of Minnesota Press, 1954.

Meehl, Paul E. “Causes and Effects of My Disturbing Little Book.” Journal of Personality Assessment, vol. 50, no. 3, 1986, pp. 370-375.

Dawes, Robyn M., David Faust, and Paul E. Meehl. “Clinical Versus Actuarial Judgment.” Science, vol. 243, no. 4899, 1989, pp. 1668-1674.

Grove, William M., and Paul E. Meehl. “Comparative Efficiency of Informal (Subjective, Impressionistic) and Formal (Mechanical, Algorithmic) Prediction Procedures: The Clinical-Statistical Controversy.” Psychology, Public Policy, and Law, vol. 2, no. 2, 1996, pp. 293-323.

Dawes, Robyn M. “The Robust Beauty of Improper Linear Models in Decision Making.” American Psychologist, vol. 34, no. 7, 1979, pp. 571-582.

Expertise, Confidence, and Its Limits:

Kahneman, Daniel, and Gary Klein. “Conditions for Intuitive Expertise: A Failure to Disagree.” American Psychologist, vol. 64, no. 6, 2009, pp. 515-526.

Oskamp, Stuart. “Overconfidence in Case-Study Judgments.” Journal of Consulting Psychology, vol. 29, no. 3, 1965, pp. 261-265.

Tetlock, Philip E. Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press, 2005.

Professions and Perceptual Authority:

Abbott, Andrew. The System of Professions: An Essay on the Division of Expert Labor. University of Chicago Press, 1988.

Freidson, Eliot. Professionalism: The Third Logic. University of Chicago Press, 2001.

Polanyi, Michael. The Tacit Dimension. University of Chicago Press, 1966.

Resistance to Algorithmic Decision Aids:

Dietvorst, Berkeley J., Joseph P. Simmons, and Cade Massey. “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err.” Journal of Experimental Psychology: General, vol. 144, no. 1, 2015, pp. 114-126.

Emanuel, Ezekiel J., et al. “Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care?” JAMA, 17 Aug. 2026.