he simplest picture of perception is a camera feeding a viewer: light comes in, signals go up, the brain looks at them. Predictive processing turns the arrows round. The brain is always guessing what it is about to sense, and what travels upward is mostly the error in the guess.

the question

Do the feedback connections in the cortex, which outnumber the forward ones in many places, carry predictions? And if they do, how far does the idea reach: perception, action, attention, mood, the self?

the classic finding

The idea goes back to Helmholtz's unconscious inference. Rao and Ballard (1999) built a model of visual cortex in which higher areas predict the activity of lower ones and lower areas send back only the difference; it reproduced odd effects in real neurons, cells that stop responding when a line runs on past their patch of the visual field. In learning, the dopamine neurons studied by Schultz, Dayan and Montague fire to rewards that are better than expected, which is a prediction error of a different kind.

Friston generalised the idea into the free-energy principle, in which perception, action and learning all reduce the gap between a model's predictions and its input [paraphrase, not a quotation]. Clark (2013) made the case to philosophers; Keller and Mrsic-Flogel proposed it as the cortex's basic computation [paraphrase, not a quotation].

what held up, and what did not

Expectation clearly changes sensory responses. Whether the brain computes explicit errors in the way the theory needs is less clear: a review of the physiology found the evidence mixed and often compatible with simpler accounts (Walsh et al., 2020). Critics also say the framework is so flexible it is hard to test (Litwin & Miłkowski, 2020), and ask why, if organisms minimise surprise, they do not sit in dark rooms; the answers are argued over (Sun & Firestone, 2020).

what it looks like in machines

A language model is trained to predict its next input. That makes it a prediction machine, but not obviously a predictive-processing one: it has no ongoing loop of prediction and correction against the world while it runs. The comparison is useful mostly for the questions it raises. See brains and models compared.

what is still open

Which cells, if any, carry the errors; whether one principle can cover so much; and whether a theory that explains everything has explained anything.

in the tour: the room called "Predictions" has a placard called "The brain as a guesser" ···

Reading list

1. Helmholtz (1867). Handbuch der physiologischen Optik. Leipzig: Voss (issued in parts from 1856; complete 1867).
2. Rao & Ballard (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience. doi:10.1038/4580
3. Schultz, Dayan & Montague (1997). A Neural Substrate of Prediction and Reward. Science. doi:10.1126/science.275.5306.1593
4. Friston (2010). The free-energy principle: a unified brain theory?. Nature Reviews Neuroscience. doi:10.1038/nrn2787
5. Clark (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences. doi:10.1017/s0140525x12000477
6. Keller & Mrsic-Flogel (2018). Predictive Processing: A Canonical Cortical Computation. Neuron. doi:10.1016/j.neuron.2018.10.003
7. Walsh et al. (2020). Evaluating the neurophysiological evidence for predictive processing as a model of perception. Annals of the New York Academy of Sciences. doi:10.1111/nyas.14321
8. Litwin & Miłkowski (2020). Unification by Fiat: Arrested Development of Predictive Processing. Cognitive Science. doi:10.1111/cogs.12867
9. Sun & Firestone (2020). The Dark Room Problem. Trends in Cognitive Sciences. doi:10.1016/j.tics.2020.02.006

see also

	prediction error — the learning-theory sense
	Hermann von Helmholtz — unconscious inference
	the nervous system — the hub