A Wharton School professor found that ChatGPT, the viral artificial intelligence chatbot, would pass a final exam in a typical Wharton MBA core curriculum class.
The professor, Christian Terwiesch, published a paper with his findings on Jan. 17, sparking a national conversation about the future of AI in education and amplifying concerns about how it can be used. The Daily Pennsylvanian sat down with Terwiesch to discuss his findings, what challenges AI presents for the future of education, and more.
Terwiesch, who currently serves as the Andrew M. Heller professor at Wharton and the chair of Wharton’s Operations, Information, and Decisions department, currently teaches MBA and executive education courses at Wharton. He is also the co-director of the Mack Institute for Innovation Management.
Terwiesch input the exam questions into the ChatGPT website to evaluate how well the responses he received from the AI would score on the exam. He told the DP that he found that ChatGPT “does an amazing job at basic operations management and process analysis questions, including those that are based on case studies.”
Terwiesch also found that ChatGPT can make simple math mistakes that are detrimental to correctly solving problems. He found that ChatGPT cannot yet effectively solve advanced process analysis questions, but that it does respond well to human hints. Based on these findings, Terwiesch wrote in his paper that the bot would receive a B to B- on his exam.
“We were uploading my first exam question into the prompt, and I was just blown away,” Terwiesch said when asked about his research.
Terwiesch described his work as an experiment rather than a research paper, explaining that he was struck with the idea to evaluate ChatGPT during winter break when discussing AI with his family. He said that there are certain types of exams that AI should not be used on: those intended to certify knowledge of a technical task, assess student mastery of a subject, and engage students with the material.
He said that there are certain types of exams that AI should not be used on: certifying knowledge of a technical task, knowing what material students like those at Wharton have mastered, and engaging the students in the material.
“For these types of tests, we cannot let the bot get involved, because we want you to be certified and not the bot,” he said.
He added that sometimes assignments are needed as instruments to engage students with material and learning processes, so using AI in these settings would be detrimental to student success.
“You suddenly get away with five minutes of work and have passed your homework, but you yourself haven't learned anything,” he said. “The answer should not be to ban it, but the answer should be how else we might engage the students.”
Echoing some students and professors who previously expressed optimism about the impact of AI, Terwiesch said that ChatGPT could have some possible benefits, including encouraging further engagement and learning: “I think ChatGPT could actually be our friend.”
“We have to find ways of taking advantage of the technology, where we find the technology as a driver of engagement as opposed to something that kills the engagement,” he added.
However, the AI's exam performance could have implications on the future of work, specifically in the healthcare industry, Terwiesch said. While many might suppose that AI would reduce the need for human labor, Terwiesch thinks otherwise.
“My prediction is actually that there is going to be more demand for healthcare,” he said. “I think for many jobs, this is going to create more work, not less work. I call this the demand-inducing effect of technology."
He explained that some of his other studies conducted with Penn Medicine found that the demand for in-office care actually increases when patients interact with physicians via digital technology.
“If I have an easy way to reach out to the doctor, I’m just going to consume more care — they cannot provide definitive care and fix the issue, so they’re going to send me back to the old process, and the net result is actually more work,” he said.
Terwiesch also does not think that AI will effectively replace human labor in the near future.
“The scenario that we will all be sitting at the beach while AI does the work, I just don’t see it. I don’t think the world is running out of work,” he said.
The Daily Pennsylvanian is an independent, student-run newspaper. Please consider making a donation to support the coverage that shapes the University. Your generosity ensures a future of strong journalism at Penn.
Donate