Meaningful perception

Understanding perception as meaning-saturated resolves several of the difficulties Part One found rationalism faces.

Recall from “Is this an eggplant which I see before me?” that the usual rationalist assumption is that perception’s job is to deliver an objective description of your physical environment. “Objective” would mean that it is independent of your theories, of your projects, and of anything that cannot be sensed at this moment, such as recent events. We saw that, for several in-principle reasons, this seems impossible.

Fortunately, that isn’t what we need from perception. In routine, practical activity, what we want perception to tell us is: what are the meaningful aspects of the situation we’re in right now? And what possibilities for ongoing activity do they suggest? The answers depend on what we know, what we can do, what we’re are up to right now, and what else is going on.

Unsurprisingly, then, scientific study of perception shows that it does not attempt to deliver objective descriptions; and shows how perception does operate on a task-dependent, contextual, meaning-saturated and knowledge-saturated basis.

The science is fascinating, and I’d love to review it in detail here, but that would take another book.¹ Instead, I will explain just enough that you can understand how routine, reasonable activity cooperates with perception to address issues that rationality unaided cannot.

I will discuss only vision, because it’s the most important human sense, and the best understood scientifically. In this chapter, we’ll return to questions posed in “Is this an eggplant I see before me?” There we asked: what is the interface between perception and rationality? Here, our question will be: what is the interface between vision and reasonable activity? What is the division of labor? And the answer will be that they are intimately entwined, with no hard boundary between them. Seeing is an aspect of doing, not a separate, encapsulated function. This implies that what we perceive is, for better or worse, inevitably affected by what we are up to at the time.

Rationality also depends on perception, of course. We use perception in building objective, rational theories. However, this use is mediated through reasonableness, which limits how objective theories can be—as we’ll see in Part Three.

In Part Three, we’ll also come to understand how the nature and limits of perception and cognition force the rather awkward ways formal rationality must work in the material world. As a hint: how much of your technical, rational work could you do if you were blindfolded? Which parts can you do without looking at a computer screen or at your lab equipment?² What does that tell you about the nature of rationality?

Seeing is routine purposeful activity

Seeing is work we do, not something that just happens. This is not obvious, because most visual activity is routine, rapid, effortless, and reliable. So, as with other routines, we don’t particularly notice what we’re doing, and instantly forget it. However, visual work is partly conscious, and can be made more so with a little attention and practice.

In rare situations, the visual task is difficult. As with other routine activities, the trouble you experience reveals something about what’s going on. The machinery’s operation becomes obvious only when it doesn’t work. So we’ll look at examples of visually difficult situations: messes and video games.

Eye movements are the most obvious, easiest to understand manifestation of visual activity. Typically, our eyes flick around several times per second, and you can become fully aware of these motions just by paying a bit of attention.³

You can see in an instant—about a quarter of a second, actually—that the photograph below shows a mess. Figuring out what it is a mess of takes a lot of work. Your eyes have to center each object separately to see what it is. You may have to stare at some items for a full second to figure out what they are. It’s somewhat unusual to feel a sense of visual effort; I’ve chosen this picture in hopes that you can experience that. You could probably spend thirty seconds on this image and still be working out some details.

This image is difficult because the objects are meaningless out of context. They have no purpose; there are no contextual clues as to why they are jumbled together like this. The deer vertebra, right at the center, would make sense as part of a skeleton, but what it is it doing here?

In a routine arrangement of familiar objects, you can take in an overview at once, and get the impression that you’ve seen the whole thing in detail; but actually you just know now where you’d need to look to see specifics, because you already know what everything means.

I forced myself to pay attention and spent more than sixty seconds looking at this photo. I kept finding new things I’d missed, some of them rather mysterious!

Learning to see

Because most of life is routine, and most objects and situations are mostly familiar, and because we’ve practiced our visual skills from early childhood, they suffice for most tasks, and go unnoticed. Needing to learn new visual skills is unusual for adults.

Some video games are an outstanding exception. Video games are designed to make learning new skills fun, and many games teach you to see in new ways. When you enter a new segment of the game, everything is happening much too fast; you have no idea where to look or what it means. Enemies come out of nowhere and kill you before you even see what they are. With practice, you learn to see things you couldn’t before, because you didn’t know how.

You are sneaking along a gloomy passageway in the necromancer’s tower. Suddenly you die. WTF just happened??

You reload the game from the last save point.

You are sneaking along that passageway and out of the corner of your eye you see something violent happen on left side of the screen and then you die. You reload.

You are sneaking along, looking left, and a golem leaps out of the archway on your left side and kills you.

This time, you’re watching the archway cautiously, and when the golem leaps out you hit it with a lightning bolt. A moment later, something happens on the right and you die.

Next try, you zap the golem with a lightning bolt, you flick your eyes right, and as the tomb over there opens, you manage to get one of the zombies with a fireball. But another one kills you. You noticed that the headless zombie hesitated for a moment before attacking.

You zap the golem and incinerate the one-arm zombie with a fireball while the headless one gropes around. You cartwheel to dodge its attack, and finish it off with a mid-air flying dagger thrust. Awesomeness! Unfortunately the floorboards you land on are rotten and you fall through to your death.

… An hour later, you stroll through the tower, knocking off monsters and skipping traps without really thinking about it, because you have learned to perceive the meanings of routine necromantic phenomena. Archways harbor golems, zombies without heads can’t see, rotten floorboards are a bit darker than solid ones. Now you know what to look out for, and where to look to see it.

You whack the necromancer, collect the Arcane Eggbeater Of Destiny, and return with it to the College of Wizards to get your next homework assignment.

Seeing with a purpose

Vision is not an “input device” like a digital camera connected to a computer. In that set-up, causality flows in only one direction, from the photons arriving at the sensor through various stages of processing to the cable connecting the camera to the computer. A camera delivers “objective” information in the sense that it’s the same regardless of what program the computer is running. In vision research, this is called bottom-up information flow. It delivers fixed information in a single format, regardless of what you want.

Extensive evidence from multiple fields shows that human vision also involves top-down information flow.⁴ Conscious reasoning processes can causally affect which visual information gets processed at the lower, pre-conscious stages, and how. Most obviously, you can choose what to look at by moving your eyes.

There are many other ways you can direct your own visual processing. One that has been studied in great detail is visual attention.⁵ If you fix your gaze direction to look at one small object for several seconds, you can still “look around the room,” transferring your attention to other objects in sequence, even if they are quite far out in peripheral vision and you can’t see them all that clearly. (Try this now!)

Experiments show that visual attention significantly affects what you see. You are much more likely to notice and identify details and events at the location you are attending to than elsewhere in your peripheral vision. Further, you are more likely to correctly identify brief events if you know what you are looking for. In the necromancer’s tower, you may learn that coffin lids twist slightly, a second before they open and a skeleton pops out. That gives you time to prepare a fireball spell, so you watch for this slight motion out of the corner of your eye.

As with other visual phenomena, the importance of attention becomes obvious only in difficult tasks. A series of experiments asked subjects to count the number of times basketballs were passed among a group of people who were also moving in complicated patterns.⁶ Because the subjects were closely attending basketball passes, they were “blind” to other information that was objectively available. The specifics of this are startling, and I don’t want to give them away. To experience the effect for yourself, I encourage you to watch a one-minute film by visual psychologist Daniel Simons, available on YouTube at https://www.youtube.com/watch?v=vJG698U2Mvo. If you are familiar with this experiment, or find it too easy, try the advanced version at https://www.youtube.com/watch?v=IGQmdoK_ZfY.

What we’ve learned from visual psychology suggests that seeing involves learned, task-specific skills. It is contextual and purposive, which makes it a good fit for everyday, reasonable, routine activity. (And not such a great fit for objective rationality.)

Visual activity is not separate from the rest of what we’re doing. The phrase “hand-eye coordination” points at this. In video games, your visual actions that tell your visual processing systems what to do are just as much part of the skill of fighting a group of monsters as swinging your sword is. Shifts of visual attention, for instance, are seamlessly integrated with the rest of the killing dance. As a more mundane example, if you are looking for scissors, you’ll move your head as well as your eyes to check around the desk, shove clutter out of the way to see behind or beneath it, and eventually get up and go open a drawer and peer inside. Visual activity and bodily motions are entangled.

Bottom-up and top-down processing interact in complex ways. For example, sudden movements get noticed automatically, bottom-up. This makes evolutionary sense in an environment with predators, prey, and potentially hostile people. In the necromancer’s tower, you saw that something was jumping out and killing you before you could see what it was.

Also bottom-up, your visual system automatically recognizes familiar objects if it’s given a few more milliseconds. Only when you see something peculiar do you need to figure out what it is by looking harder, more deliberately.

Seeing with an ontology

Much of what you see, you see as something. You don’t see a textured black region of the visual image, you see a loudspeaker. Or a raven. It’s already a loudspeaker or a raven when you first experience it.⁷ Bottom-up vision has done that work for you.

What you see something as depends on your knowledge, context, and purposes. If you are familiar with moussaka and you see it on a plate in a restaurant, you’ll probably see it as moussaka. If you aren’t, you’ll probably see it as a mushy casserole. You can’t see it as moussaka, because that’s not part of your ontology. If you see moussaka on a city sidewalk, you might just see it as disgusting, potentially pathogenic slop that you want to avoid. What you see a clump of atoms as depends on what you are looking out for, and why. Although bottom-up processes can do much of the work for you, especially in the case of rigid manufactured objects like loudspeakers with standard shapes and colors, your top-down direction often also plays a critical role.⁸

Because perception evolved to enable purposeful activities, most of the time it reveals meaningful functions and potentials. Those are a matter of ontology: not just categories, but also how you separate the world into objects; what properties you see them having; and how they relate to each other. It’s not just objects, but also intentions, actions, events, environments, and possibilities. In context, you see someone kneeling as worship; someone running out of a store as shoplifting; and items on a restaurant table as a dinner setting, not a silverware collection. And, as in Simons’ films, you may simply never see things that aren’t meaningful.

Routine activity is easy because most of the time we can see what to do. We see affordances—cues as to what actions are possible, and what their effects will be. When it’s time for a snack, looking around inside the refrigerator reveals what you feel like eating. Noticing a coffin-lid twisting is inseparable from recognizing the danger and the need to get a spell ready. We can, in effect, see into the future.

Logical positivism sought to eliminate the “theory-ladenness” of perception, because it biases evidence. That’s impossible, but for most purposes it’s also undesirable. Usually, seeing meaning is useful, and not a mistake. Sometimes, however, it does skew our understanding, and stripping meaning helps. We’ll see in Part Three that this is a function of technical rationality, and we’ll explore some ways of accomplishing it.

Seeing nebulosity

Perception is inherently nebulous, and you can’t fully denebulize it.

You can see at a glance that this is a picture of a speckled hen.⁹ But how many speckles can you see? Certainly more than ten, and less than a thousand. If you wanted a more precise number… you’d have to count them individually, shifting your visual attention to each in turn. So what do you perceive before doing that? A nebulous number of speckles.¹⁰

Look closer… some speckles are quite well-defined, but others fade out gradually, run into each other, or are not clearly speckles at all. You cannot count them after all. The number is itself nebulous. Perception is nebulous because reality is nebulous. This is an ontological issue, not an epistemological one. You are prevented from knowing the number of speckles not because just your perceptual abilities are limited and unreliable, but because there is no definite number.

You need to see precisely enough only to accomplish your task. If you spill coffee on the carpet, it makes a splotch with an indefinite boundary. The puddle fades around the edges as wicking action transports the liquid along fibers. You cannot see exactly where the mess ends. Fortunately, it doesn’t matter. You mop it up with paper towels, and then soak it with hot water from a sponge, and then dry it with paper towels again. You don’t have to cut the towels to precisely fit the shape of the splotch. You can just press a wad of sheets down on the floor to cover the general area.

It seems that we have access to perceptual processing at many different abstraction levels. There is no crisp, fixed “neutral observation vocabulary,” as the logical positivists hoped, because there are few if any objective, non-nebulous macroscopic properties to be perceived.

Ravens are black. Or are they?

If you pay closer attention to the color, you can see it more accurately. It helps to get physically closer to the bird, but even at a constant distance, visual effort makes a difference. You may be able to see that some parts of the bird are dark matte gray. Others are iridescent with purple, blue, and green highlights. As your vision penetrates the depths of the colors, they may come to seem intense and vivid. How could you have ever have seen ravens as black?

To get an objective measurement of color, you can use a spectrophotometer. The instrument’s use is quite limited, however. The spectrum reflected from an object depends on the incoming light. That must be controlled by putting the object in an opaque box to shield it from random lighting, and illuminating it with a well-characterized source instead. Further, “an object” does not have a color; remember the speckling of the hen and the streakiness of an apple. In practice, spectrophotometers are used only to measure small samples of homogeneous materials, typically chemicals that you have just made.

Many professions develop specialized skills of seeing. This includes science. In Part Three, we’ll look at an example of a chemistry professor teaching graduate students to judge when to stop a reaction by closely observing its exact quality of blackness.