Penn’s Computational Social Science Lab recently launched a tool to measure the concept of common sense.
The CSSLab is currently collecting feedback on a common sense survey with the goal of implementing it into artificial intelligence language models and other applications. It aims to recruit responses from as many people as possible to detect systematic inconsistencies within common-sense beliefs.
Penn researcher Mark Whiting and Annenberg School for Communication professor and CSSLab Director Duncan Watts published research in January that presented a framework for quantifying common sense. Their work focused on better understanding how humans perceive common sense in both individual and collective sense.
“There’s a complicated relationship between the common beliefs we share and those we don’t,” Whiting said. “For things like voting for a president, is there really a one-size-fits-all solution?”
Alongside a team of undergraduate and graduate researchers, Whiting and Watts created a mechanism for identifying how common sense is conceptualized by individuals and the greater public. The team defined this perception as “commonsensicality” through a series of claim-based tests. This analysis then allowed the researchers to tie together threads of common understanding across groups of people.
According to Whiting, the team arrived at their primary methodology by using both AI-generated and human-generated claims, as well as information from existing databases. He said that the “core technique” centered around asking people what they think about a claim and if they believe others will think the same. If many participants agree on a claim’s truthfulness and also believe that others agree, the claim receives a high commonsensical score, Whiting explained.
“We tend to overlook the fact that there’s a huge cultural or cross-person variation when it comes to common sense interpretations,” Penn doctoral candidate and CSSLab researcher Josh Nguyen said.
Nguyen emphasized that the answer to a question can vary based on personal context, as different experiences or circumstances shape individual interpretations. These differences in what people perceive as common sense create problems when integrating that knowledge into AI language models.
“The illusion of machine common sense seems to be illuminated by the fact that there are a lot of AI benchmarks for common sense that these models tend to pass really well. But the implication of passing that exam is actually more complicated because it depends on who you’re asking,” Nguyen said.
Whiting noted that people often assume advanced systems such as ChatGPT truly comprehend situations and possess intelligence. However, he questioned whether these systems actually have common sense and if they match human understanding.
By centering the study on common beliefs within and across different groups of people, the team intends to apply their research to AI while still focusing on the human aspects behind the concept of common sense.
Engineering first year Julie Sidana said that the most interesting element of the research was the demographic of the sample size.
“Who are they testing — is it just Penn students? Are they looking at a lot of different ages? This seems qualitative, so I’m curious how they are quantifying it,” Sidana said.
The CSSLab is encouraging all Penn students, as well as anyone else interested, to participate in the survey.
The Daily Pennsylvanian is an independent, student-run newspaper. Please consider making a donation to support the coverage that shapes the University. Your generosity ensures a future of strong journalism at Penn.
Donate
