Global Courant
A team of researchers from four Canadian and American universities says artificial intelligence could replace humans when it comes to collecting data for social science research.
Researchers from the University of Waterloo, University of Toronto, Yale University and the University of Pennsylvania published an article in the magazine Science on June 15 on how AI, especially large language models (LLMs), can influence their work.
“AI models can represent a wide variety of human experiences and perspectives, potentially giving them a greater degree of freedom to generate diverse responses than conventional methods from human participants, which may help mitigate generalizability issues in research,” says Igor Grossmann , professor of psychology at Waterloo and a co-author of the paper, said in a press release.
In fact, Philip Tetlock, a professor of psychology at UPenn and co-author of the paper, says LLMs will revolutionize human-based forecasting in just three years.
In their article, the authors ask the question: “How can social science research practices be adapted, even reinvented, to harness the power of fundamental AI? And how can this be done while ensuring transparent and replicable research?”
The authors say that the social sciences have traditionally relied on methods such as questionnaires and observational studies.
But with LLMs’ ability to sift through vast amounts of text data and generate human-like responses, the authors say this presents a “new” opportunity for researchers to test theories of human behavior more quickly and on a much larger scale.
Scientists could use LLMs to test theories in a simulated environment before applying them in the real world, the paper says, or to gather different perspectives on a complex policy issue and come up with possible solutions.
“It makes no sense for people to venture probabilistic judgments into serious policy debates without the help of AIs. I estimate a 90 percent chance of that,” Tetlock said. “Of course, how people react to all that is another matter.”
However, one problem the authors identified is that LLMs often learn to rule out sociocultural biases, which raises the question of whether models correctly reflect the populations they study.
Dawn Parker, a University of Waterloo professor and paper co-author, suggests that LLMs be open source so that their algorithms and even data can be checked, tested or modified.
“Only by maintaining transparency and replicability can we ensure that AI-enabled social science research really contributes to our understanding of the human experience,” said Parker.
