What probability can’t do

If probability theory actually was an epistemology, what would we want it do?

What we’d want would be a method that, given a set of observations and a candidate belief, tells us how strongly we should believe it. That would be a solution to the problem of induction.¹ Probabilists assume that such a method must exist—although different schools of probabilists believe in different methods.

The method should be mechanical or mindless, in the sense that it requires no fallible human judgement. This might make a guarantee possible: if you push your data through the process, it will tell you the mathematically correct degree of confidence you should have in your candidate belief. Probabilism recognizes that certainty about specific beliefs is impossible, but still promises certainty that its results are optimal.

Unfortunately, no mathematical method can reliably tell you how confident you should be in a belief; the desired guarantee is impossible. Further, the belief that such methods and guarantees do exist has been a major cause of the 2008 financial crisis and the science replication crisis, among other catastrophes.

If you use probabilistic methods in your professional practice, you have a moral responsibility to understand how and when and why they work—and how and when and why they don’t—to avoid such failures. The stakes are high. Millions of people have died due to probabilistic rationality not having been used where it should have been, in public health and medical decision making. Trillions of dollars—equivalent to millions of people’s entire working lives—have been wasted due to using probabilistic rationality where it should not have been, in financial market modeling.

Unfortunately, understanding when to use probabilistic rationality is uncommon. Rationalist assurances that it is always the right tool in situations of uncertainty are largely to blame.

So… why doesn’t probabilistic rationality always work? In short, because the world is not a casino.

There are a couple dozen different ways most situations are unlike casinos, each of which makes probabilistic rationality inapplicable in a different way. Any one of these is fatal to probabilism as a general theory of rationality.

The next chapter sketches, intuitively, one reason no probabilistic method could work as desired. It’s the simplest and most important one: possible actions and outcomes are nebulous and unenumerable, so the formalism simply does not apply in most real-world situations.

Then the following chapter explains the science replication crisis as partly a consequence of probabilism.

But, we can come to a happier understanding if we turn the question around. Why does probabilistic rationality sometimes work, even though it is never formally applicable in the eggplant-sized world?

Skilled meta-rational use of probabilistic methods requires considering whether and how a situation is enough like a casino. If you judge it is, you need to take the specifics of the similarities and dissimilarities into account in your use of a probabilistic model. Part Five includes a chapter on such meta-rational probabilistic and statistical practice.

Footnote for rationalists

This chapter was supposed to be about “what probability can’t do,” but you haven’t given even one specific example! This is a lot of words—just philosophical hot air. You can’t prove probability theory is wrong, because it’s just math. It’s just true; you can’t argue with math!

Probability theory is just math, but probabilism isn’t. Probabilism is about applying the math to understand eggplant-scale, real-world situations.² The math is fine; it’s the application that won’t always work. In fact, no application is exactly right, because you can never specify everything that might happen.

So, demanding “a specific example” misses the point: everything is an example. However, Leonard Savage, founder of the Subjective Bayesian school of statistics, provided one. He said that using probabilistic methods to plan a picnic would be “utterly ridiculous,” because the set of possible consequences of actions cannot be known in advance.³

Probabilists offer several supposed proofs that probability theory is guaranteed to be right. These are mathematically correct but mostly irrelevant to real-world problems of induction. The most common ones are the Dutch Book Argument and Cox’s Theorem.

• The Dutch Book Argument shows that if a formal system conforms to axioms describing fair betting games, then the bets that probabilistic rationality recommends are optimal.⁴ This is irrelevant except in situations you judge are similar enough to a formally fair betting game.
• Cox’s Theorem says that there is no formal system other than probability theory that is very similar to it; so if you want something like that, you’ve only got one choice.⁵ This is irrelevant unless you are considering using one of the dubious alternatives, none of which seems to work as well in practice.

I mentioned that there are “a couple dozen” fatal flaws in probabilism. I have not found any unified, comprehensive survey of these.

• “Bayes: a kinda-sorta masterpost,” an internet essay by “Nostalgebraist,” explains clearly some of the failings I consider most important.
• The “Potential Problems” section of the Stanford Encyclopedia of Philosophy article on “Bayesian Epistemology” covers eight briefly.
• John Earman’s Bayes or Bust? covers several arguments for and against. He concludes that probabilism doesn’t work and can’t be fixed, but that alternative theories of induction are even worse.
• Nassim Nicholas Taleb has written several books about the mistaken assumption that reality is a casino, which he calls the “ludic fallacy.”

Most critics of probabilism have analyzed a single defect. Probably no one has collected a full list because there’s so many of them; because each takes several pages to explain well; and because if you want to argue against probabilism, a single conclusive argument seems sufficient. Individual fatal flaws do not seem to deter probabilists, however. They have devised conceptual work-arounds, technical add-ons, or philosophical counter-arguments for many of them. I find these weak individually, and it’s hard to imagine that they could all be combined into a unified system.

A comprehensive collection of reasons probabilism can’t work, together with the proposed work-arounds, could be helpful. It would be a gigantic project, though. Adding to the complexity, there are several schools of probabilism, all agreeing that the methods and philosophical defenses of the other schools don’t work.

As I observed in “Rationalism’s responses to trouble”:

Eventually one just has to say “This contraption has gotten awfully complicated, and mostly doesn’t seem to work in practice. Perhaps you will be able fix it someday with even more machinery, but that seems increasingly unlikely. And we do have a better alternative!”