hen a person and a model work together, the hope is that the pair does better than either alone. That outcome has a name, complementarity, and the most important recent finding about it is how often it fails to happen.

the question

Licklider (1960) imagined "man-computer symbiosis": people setting goals and judging results, machines doing the routine work that prepares the way for insight. Horvitz (1999) set out design principles for mixed-initiative interfaces, in which the system decides when to act, when to ask, and when to stay out of the way. The empirical question is when such partnerships actually beat the better partner working alone.

the findings

Vaccaro, Almaatouq & Malone (2024) meta-analysed experiments that measured humans alone, AI alone, and the combination. On average the combination performed worse than the better of the two. The picture differed by task: losses were concentrated in decision tasks, such as classifying or judging, while creation tasks, such as writing, showed gains. Combinations tended to help when the human alone outperformed the AI, and to hurt when the AI was better.

Explanations do not reliably rescue this. Bansal et al. (2021) found that AI explanations increased people's acceptance of the AI's recommendation whether or not it was correct, so they did not produce complementary performance. Buçinca, Malaya & Gajos (2021) tried "cognitive forcing" designs, such as asking for a decision before showing the AI's, and found they reduced overreliance, though people liked them less. Steyvers et al. (2022) showed in a Bayesian model and data that combining human and machine judgments can be complementary even when the two differ in accuracy, provided their errors are not the same and confidence is used well.

field evidence

Outside the lab, results look better for some kinds of work. Noy & Zhang (2023) found in a preregistered experiment that access to a chat model cut the time professionals took on writing tasks and raised rated quality, with the largest benefits for weaker writers. Brynjolfsson, Li & Raymond (2025) studied customer-support staff given an AI assistant: productivity rose on average, by much more for novice and less-skilled workers, and very little for the most experienced. Costello, Pennycook & Rand (2024) found that short conversations with a language model reduced belief in conspiracy theories, with effects lasting months, a case where the model did persuasive work people often fail at.

reading these together

The lab and field results are not as contradictory as they look. Field studies mostly measure speed and quality on creation-like work, where the meta-analysis also found gains. Lab studies mostly measure accuracy on decisions, where people tend either to defer too much (automation bias) or too little. Complementarity seems to need differences in what each partner gets wrong, and a way for the person to know when to override.

live debates

When does the team win? Dell'Acqua et al. (2023) describe a jagged frontier: consultants using a model did better on tasks inside its capabilities and worse on a task outside them. Agarwal et al. (2023) found that radiologists given AI predictions did not improve on average, and modelled why: they underweighted the AI and treated its signal as if independent of their own [paraphrase, not a quotation]. Green & Chen (2019) found that people using a risk assessment deviated from it in ways that introduced disparities. Donahue, Chouldechova & Kenthapadi (2022) give theoretical conditions under which complementarity is possible at all.

methodological critiques

Complementarity must be scored against the better of human alone and model alone; many studies compare only against the human. Lai et al. (2023) found wide variation in tasks, participants and incentives across the empirical literature, much of it short tasks with crowd workers; Bansal et al. (2021) showed explanations can raise acceptance of AI advice whether it is right or wrong.

the open problem

No one yet has a reliable recipe for complementarity in decision tasks. The candidates, calibrated confidence, forcing a first guess, and routing cases by who is likely to be right, each help somewhere. Which works for which task is the current frontier.

Reading list

1. Licklider (1960). Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics. doi:10.1109/THFE2.1960.4503259
2. Horvitz (1999). Principles of mixed-initiative user interfaces. Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99. doi:10.1145/302979.303030
3. Vaccaro, Almaatouq & Malone (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour. doi:10.1038/s41562-024-02024-1
4. Bansal et al. (2021). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3411764.3445717
5. Buçinca, Malaya & Gajos (2021). To Trust or to Think. Proceedings of the ACM on Human-Computer Interaction. doi:10.1145/3449287
6. Steyvers et al. (2022). Bayesian modeling of human–AI complementarity. Proceedings of the National Academy of Sciences. doi:10.1073/pnas.2111547119
7. Noy & Zhang (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science. doi:10.1126/science.adh2586
8. Brynjolfsson, Li & Raymond (2025). Generative AI at Work. The Quarterly Journal of Economics. doi:10.1093/qje/qjae044
9. Costello, Pennycook & Rand (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science. doi:10.1126/science.adq1814
10. Dell'Acqua et al. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. SSRN working paper (Harvard Business School Technology & Operations Mgt. Unit Working Paper 24-013). doi:10.2139/ssrn.4573321
11. Agarwal et al. (2023). Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology. NBER Working Paper 31422. doi:10.3386/w31422
12. Green & Chen (2019). Disparate Interactions. Proceedings of the Conference on Fairness, Accountability, and Transparency. doi:10.1145/3287560.3287563
13. Donahue, Chouldechova & Kenthapadi (2022). Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness. 2022 ACM Conference on Fairness Accountability and Transparency. doi:10.1145/3531146.3533221
14. Lai et al. (2023). Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies. 2023 ACM Conference on Fairness Accountability and Transparency. doi:10.1145/3593013.3594087

see also

	J. C. R. Licklider — symbiosis, 1960
	annotated: human–AI interaction — every paper on this page, and more
	automation bias — the commonest way a team fails
	glossary: complementarity
	swarms — many simple parts, one better whole