The probability of green cheese

Recall McCarthy’s rowboat problem. Usually you can use one to cross a river. But, when you get there, you might discover it’s impossible due to an unexpected infestation of inquisitors.

You could interpret “it’s usually true that you can use a rowboat to cross a river” as “it’s probably true,” in the technical sense according to which “probably” means you assign it a number between zero and one. How?

The mathematics of probability starts from a specified set of possible “outcomes.” The probabilities of all outcomes must add up to 1.0 (or else the math doesn’t work at all). You could treat “I can cross the river” as an outcome, and assign it some probability somehow,¹ like maybe 0.9.

So what’s the probability of a missing oar? I don’t know, 0.01 maybe? Hole in the boat? Same-ish? Breaking an arm? Less probable: say 0.001. Electric eels? Ebola epidemic? Invasion of space aliens?

The problem I want to point at is not that this exercise adds up a list of pretty meaningless numbers to get an even more meaningless one—although that does seem to make probabilistic reasoning useless here.

The problem is that there are unenumerable unknowns. Unlike in a formal game, you can’t make a full list of possible outcomes. There are always more things that might happen. If you add up your estimated probabilities, you’ll eventually exceed 1.0.² Alternatively, if you quit before running out of ideas, you will have underestimated the total probability of failure.

And these are just known unknowns. There are also possibilities no one would ever think of, because we don’t know everything about how the world works.

So, a sensible strategy is to lump all outcomes you haven’t thought of as “something else happens,” and estimate the probability of that. This is very like the logicist closed-world idealization approach: you just pretend you have made a complete list, and proceed. In statistics, this is called a “small-world idealization.”³

Using probabilistic rationality in the eggplant-sized world always requires a small-world idealization. By excluding an unknown set of unaccounted-for factors, that always risks making the analysis so wrong that it is worse than useless. The “other” category is, actually, a probabilistic model of the entire rest of the universe.⁴

If you use a rowboat regularly, your probability estimates from experience may be good enough for the job. Science explores realms unknown and un-understood. Initial probability estimates must often be meaningless, by definition.

Here one may reasonably ask why we are in the business of “estimating” probabilities at all. That’s inherently subjective and unscientific. Can’t we just let the data do the talking? As we make more and more observations, we’ll get increasingly accurate probabilities. Statistical methods can tell us how close to the true values they are. Everyone learned this in high school science class.

This line of thinking is disastrously wrong, as the next chapter explains. Let’s do a thought-experiment first, to get the intuition.

Suppose you are an astronomer in charge of a telescope with an extra-fancy spectrometer. You arrive at work one evening, and read an email from a colleague on a different continent: “We’ve just seen something really odd in lunar crater X9-PDQ-17b. Some kind of explosion; can’t make any sense of it. Could you check it out with your spectrometer asap?” So you look up the coordinates of X9-PDQ-17b—which turns out to be dinky, only a few dozen meters across—and you tell the telescope to look there.

A spectrometer breaks light down into its constituent wavelengths: a spectrum, as with a prism. A wavelength corresponds to a pure color. Sunlight is a mixture of many. Particular materials—such as rocks on the moon—reflect different percentages of incoming sunlight at different wavelengths. This gives each a unique “signature.” A spectrometer measures precisely how bright light is at each wavelength.

Your spectrometer is at the optical focus of the telescope, so it tells you what you are looking at. Its software consults a large public database of spectra of all sorts of materials. Each entry says what the material is, and how much it reflects each wavelength. Entries are uploaded by expert spectroscopists, who have rigorously calibrated their instruments, and made and cross-checked many measurements.

Typically, an observed spectrum doesn’t precisely match anything in the database. Using a statistical algorithm, the software calculates how nearly it fits every entry, and then tells you how confident you can be that you are looking at, for example, olivine, ilmenite, or anorthosite—typical moon rocks.⁵

So now it’s finished its measurement of X9-PDQ-17b, and says “green cheese, with probability 0.9837.”

What should you believe is the probability that X9-PDQ-17b is made out of cheese?

Very large amounts of data, plus a sophisticated, widely-used, carefully validated statistical algorithm, say it’s 0.9837. Do you believe that?

No? Why not? If you think the probability is very nearly zero—how come?

It occurs to you that the telescope was cleaned during the daytime today, before you came in. Maybe the cleaner had a green cheese sandwich for lunch, and a crumb fell onto a lens? Green cheese in the spectrometer seems way more likely than green cheese in X9-PDQ-17b.

The spectrometer’s probability estimate implicitly relies on a small-world idealization that excludes unenumerable ways data may not tell you what it usually does. “There’s no extraneous matter in the light path” is a usualness condition that the software doesn’t take into account. What is the probability of there being green cheese on a lens? Who knows; you couldn’t get a meaningful estimate. It’s meant to be excluded by shielding—the spectrometer is enclosed in a metal box—but maybe its shutter was mistakenly left open during cleaning.

On the other hand… it would be an odd coincidence for your colleague to have sent you an urgent email just at the same time the usualness condition was violated. “Oh, of course!” you realize. “It’s April Fools’ Day. Somehow every year I forget, and get taken in by something.” Someone on your team must have altered the software to say “green cheese,” and colluded with the foreign colleague to pull your leg. Now it seems that the probability of green cheese on the moon or on a lens is slight.

But then… you remember that, driving up the mountain to the telescope, you were listening to the radio; and there was an interview with a spokesperson from the Chinese space agency, who said that Western scientists should expect a “special surprise,” demonstrating new Chinese capabilities.

“OMG!” you think. “Could they have?… Surely not… But what an brilliant prank if they did! Explode a green cheese bomb, scattering a thin film of the material across the crater? As an April Fools’ joke?”

Now what do you think the probability of green cheese in X9-PDQ-17b is? Maybe the spectrometer was right! It’s hard to put a meaningful number on the probability, but it seems more likely than green cheese having fallen out of someone’s sandwich.

So, why did you not believe the 0.9837 estimate in the first place? Because you know that the moon is not made out of green cheese; and because you know that small-world idealizations can fail.

A meta-rational approach to probability considers the specifics of the idealization, asking whether it’s sensible; and frequently reality-checks the conclusions of probabilistic inferences, because there can be no guarantee that they aren’t wildly wrong.