rom a man in a box to models tuned on approval: machines (and one horse) that people asked questions of, and what they learned by asking. In the tour there is a room of machines that answer, with a kiosk; its exhibits belong to the tour, and the plain tour lists them.

1770	Kempelen builds the Turk, a chess automaton with a hidden human player (Standage, 2002).
1821	Willis argues from its dimensions that a person hides in the Turk.
1836	Poe's essay "Maelzel's Chess-Player" reaches the same conclusion by other arguments.
1854	The Turk is destroyed by fire in Philadelphia.
1904	A Berlin commission finds no trickery in Clever Hans.
1907	Pfungst shows Hans reads unintended cues from his questioners (English translation 1911).
1943	McCulloch & Pitts: a logical calculus of neuron-like units.
1950	Turing proposes the imitation game. See Turing.
1956	Newell & Simon: the Logic Theorist proves theorems. See Simon.
1958	Rosenblatt: the perceptron, a machine that learns. See Rosenblatt.
1960	Licklider: man–computer symbiosis.
1966	Weizenbaum: ELIZA. See Weizenbaum and try it on the minds page.
1971	Colby et al.: PARRY simulates paranoia, and psychiatrists try to tell it from patients.
1972	PARRY and ELIZA's DOCTOR talk to each other over the ARPANET; the transcript is published as RFC 439.
1994	Nass, Steuer & Tauber: computers are social actors.
2005	A crowd-work service is named "Mechanical Turk", after the automaton.
2017	Christiano et al.: learning a reward from human preference comparisons.
2022	Ouyang et al.: instruction-following models tuned with human feedback.
2023	Perez et al. and Sharma et al. document sycophancy in such models.

Sources

1. Standage (2002). The Turk: The Life and Times of the Famous Eighteenth-Century Chess-Playing Machine. New York: Walker.
2. Willis (1821). An Attempt to Analyse the Automaton Chess Player of Mr. de Kempelen. London: J. Booth.
3. Poe (1836). Maelzel's Chess-Player. Southern Literary Messenger, April 1836.
4. Pfungst (1911). Clever Hans (the Horse of Mr. von Osten), trans. C. L. Rahn. New York: Henry Holt. German original 1907. Free edition at Project Gutenberg. www.gutenberg.org/ebooks/33936
5. McCulloch & Pitts (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics. doi:10.1007/BF02478259
6. Turing (1950). Computing machinery and intelligence. Mind. doi:10.1093/mind/LIX.236.433
7. Newell & Simon (1956). The logic theory machine--A complex information processing system. IEEE Transactions on Information Theory. doi:10.1109/TIT.1956.1056797
8. Rosenblatt (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review. doi:10.1037/h0042519
9. Licklider (1960). Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics. doi:10.1109/THFE2.1960.4503259
10. Weizenbaum (1966). ELIZA—A Computer Program For the Study of Natural Language Communication Between Man and Machine. Communications of the ACM. doi:10.1145/365153.365168
11. Colby, Weber & Hilf (1971). Artificial Paranoia. Artificial Intelligence. doi:10.1016/0004-3702(71)90002-6
12. Cerf (1973). PARRY encounters the DOCTOR. RFC 439, Network Working Group (a transcript of a session on 18 September 1972). www.rfc-editor.org/rfc/rfc439
13. Nass, Steuer & Tauber (1994). Computers are social actors. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. doi:10.1145/191666.191703
14. Christiano et al. (2017). Deep reinforcement learning from human preferences. arXiv:1706.03741 (NeurIPS 2017). arxiv.org/abs/1706.03741
15. Ouyang et al. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155 (NeurIPS 2022). arxiv.org/abs/2203.02155
16. Perez et al. (2023). Discovering Language Model Behaviors with Model-Written Evaluations. Findings of the Association for Computational Linguistics: ACL 2023. doi:10.18653/v1/2023.findings-acl.847
17. Sharma et al. (2023). Towards understanding sycophancy in language models. arXiv:2310.13548 (published at ICLR 2024). arxiv.org/abs/2310.13548

see also