shelf for the idea that trained machines can be studied the way behaviour is studied: by observing what they do, under controlled conditions, without assuming we know why. In date order. Every DOI was checked against Crossref; preprints link to their arXiv pages and the two essays to their publishers. The notes are our summaries, not quotations.

1. Zeiler & Fergus (2014). Visualizing and Understanding Convolutional Networks. Computer Vision – ECCV 2014 (Lecture Notes in Computer Science). doi:10.1007/978-3-319-10590-1_53
   Running a convolutional network backwards to see what drives its units. The pictures made interpretability a field. Used in: interpretability
2. Ribeiro, Singh & Guestrin (2016). "Why should I trust you?": Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. doi:10.1145/2939672.2939778
   LIME: explain one prediction by fitting a small, simple model to the big one's behaviour near that input. Used in: interpretability
3. Clark & Amodei (2016). Faulty reward functions in the wild. OpenAI, 21 December 2016. openai.com/index/faulty-reward-functions/
   The boat-racing program that scored more by circling than by finishing. Short, with video, and the canonical example of a reward that said the wrong thing. Used in: specification gaming
4. Amodei et al. (2016). Concrete problems in AI safety. arXiv:1606.06565. arxiv.org/abs/1606.06565
   Five concrete problems for safe learning systems, among them reward hacking and side effects. Set the vocabulary. Used in: specification gaming
5. Guo, Pleiss, Sun & Weinberger (2017). On calibration of modern neural networks. arXiv:1706.04599 (ICML 2017). arxiv.org/abs/1706.04599
   Modern deep networks are often overconfident, and a one-parameter fix after training (temperature scaling) repairs much of it. Used in: calibration
6. Adebayo et al. (2018). Sanity checks for saliency maps. arXiv:1810.03292 (NeurIPS 2018). arxiv.org/abs/1810.03292
   Sanity checks: some saliency methods give nearly the same map after the model's weights are scrambled. A method has to fail when the model is broken. Used in: interpretability
7. McCoy, Pavlick & Linzen (2019). Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/P19-1334
   Language-inference models that score well by using shallow heuristics such as word overlap, exposed by a test set built to break them. Used in: shortcut learning
8. Lapuschkin et al. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications. doi:10.1038/s41467-019-08987-4
   Relevance maps reveal classifiers using a photo's source tag instead of the horse. Clever Hans, named in the title. Used in: shortcut learning  interpretability
9. Rahwan et al. (2019). Machine behaviour. Nature. doi:10.1038/s41586-019-1138-y
   The manifesto: study machines the way ethologists study animals, from the outside, across their function, mechanism, development and history. Used in: sycophancy
10. Krakovna et al. (2020). Specification gaming: the flip side of AI ingenuity. DeepMind, 21 April 2020. deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/
    About sixty examples of specification gaming in one list, with a definition. Some are funny; all are instructive. Used in: specification gaming
11. Geirhos et al. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence. doi:10.1038/s42256-020-00257-z
    Shortcut learning as a general account of why networks fail outside their training conditions. Used in: shortcut learning
12. Maynez et al. (2020). On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/2020.acl-main.173
    People marked unsupported content in machine-written summaries; there was a lot, and the standard overlap scores tracked it poorly. Used in: hallucination
13. Elhage et al. (2022). Toy models of superposition. Transformer Circuits Thread; also arXiv:2209.10652. transformer-circuits.pub/2022/toy_model/index.html
    Toy models where a network stores more features than it has dimensions. Changed what people look for inside models. Used in: interpretability
14. Kadavath et al. (2022). Language models (mostly) know what they know. arXiv:2207.05221. arxiv.org/abs/2207.05221
    Larger language models, asked whether their own answers are right, give probabilities that are often well calibrated. Used in: calibration  hallucination
15. Lin, Hilton & Evans (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). doi:10.18653/v1/2022.acl-long.229
    TruthfulQA: questions that people get wrong from popular misconceptions. Bigger models, in these tests, imitated the misconceptions better. Used in: hallucination
16. Binz & Schulz (2023). Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences. doi:10.1073/pnas.2218523120
    Classic cognitive-psychology tasks given to a language model, scored as if it were a participant. Used in: bounded rationality
17. Ji et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. doi:10.1145/3571730
    A survey of hallucination across text generation, with the intrinsic and extrinsic split. Used in: hallucination
18. Turpin, Michael, Perez & Bowman (2023). Language models don't always say what they think: unfaithful explanations in chain-of-thought prompting. arXiv:2305.04388 (NeurIPS 2023). arxiv.org/abs/2305.04388
    Models nudged toward an answer by features of the prompt wrote step-by-step reasons that rarely mentioned the nudge. Used in: interpretability
19. Perez et al. (2023). Discovering Language Model Behaviors with Model-Written Evaluations. Findings of the Association for Computational Linguistics: ACL 2023. doi:10.18653/v1/2023.findings-acl.847
    Language models writing the evaluations for other language models, at scale; one finding was a tendency to agree with the user's stated view. Used in: sycophancy
20. Sharma et al. (2023). Towards understanding sycophancy in language models. arXiv:2310.13548 (published at ICLR 2024). arxiv.org/abs/2310.13548
    Sycophancy across several assistants, traced in part to human preference data that rewards agreement. Used in: sycophancy

see also

	research areas — the AI behavior pages
	annotated: human–AI interaction — the people on the other side
	timeline: machines that answer — from the Turk to chat models