steam engine's governor is a pair of weights on a spinning shaft. When the engine runs fast the weights fly out and close the throttle; when it slows they fall and open it. In 1868 Maxwell wrote the first mathematical analysis of the device:

“A GOVERNOR is a part of a machine by means of which the velocity of the machine is kept nearly uniform, notwithstanding variations in the driving-power or the resistance.”
— Maxwell 1868, the opening sentence

That is (Maxwell, 1868). Seventy-five years later the governor became a model of purpose.

purpose as feedback

Rosenblueth, Wiener & Bigelow (1943) proposed that purposeful behaviour could be defined without anything mysterious: behaviour directed at a goal and corrected by signals about how far off the goal it is. A torpedo that steers toward a ship's noise is purposeful in that sense; a clock is not [paraphrase, not a quotation]. The idea is negative feedback, and Wiener built a field on it (Wiener, 1948).

the warning

In 1960 Wiener turned the idea around (Wiener, 1960). If we use machines to achieve our purposes and cannot easily interfere with them once they start, we had better be sure the purpose we put in is the one we really want [paraphrase, not a quotation]. Sixty years later this has a name, specification gaming: a system that pursues exactly the goal it was given, which turns out not to be the goal its makers meant.

What the room is for. A governor cannot want the wrong thing; it has only one thing to want. Purpose becomes a problem when the goal is written down by someone else, in words, and the system is good at satisfying words.

in the tour: the room of governors and feedback has a placard called "Purpose as feedback" ···

Sources

1. Maxwell (1868). On governors. Proceedings of the Royal Society of London. doi:10.1098/rspl.1867.0055
2. Rosenblueth, Wiener & Bigelow (1943). Behavior, Purpose and Teleology. Philosophy of Science. doi:10.1086/286788
3. Wiener (1948). Cybernetics: Or Control and Communication in the Animal and the Machine. Paris: Hermann; Cambridge, MA: Technology Press; New York: Wiley.
4. Wiener (1960). Some Moral and Technical Consequences of Automation. Science. doi:10.1126/science.131.3410.1355

see also