COVID-19 has limited impact on Penn Course Review ratings, DP analysis finds

Course quality and difficulty ratings were not significantly impacted by the COVID-19 pandemic despite the switch to online learning.

Penn Course Review quality ratings mostly increased across the undergraduate schools during the fall and spring semesters, but declined in summer 2020. The changes were not significantly different from patterns in previous years. Professors and students attributed the largely stable ratings to the resilience of the Penn community and creative measures faculty members took to move their coursework online. 

The Daily Pennsylvanian Analytics Staff conducted an analysis of course ratings between the Spring 2018 semester and the Fall 2020 semester, including summer sessions. The data has been aggregated from end-of-semester Penn Course Evaluations and includes attributes such as instructor ratings, course quality ratings, and course difficulty ratings. 

Course quality during COVID-19

COVID-19 forced all courses to transition to online formats in March 2019 when Penn required all students to leave campus to limit the spread of the pandemic. Despite the transition to online learning, the impact on course quality ratings has been limited. 

Wharton and Engineering courses generally rank lower in quality than Nursing and College courses. All of the schools except for Nursing typically have higher quality rankings during the summer session. In summer 2020, however, quality fell across College, Nursing, and Engineering courses, but increased across Wharton courses.

Director of the Center for Teaching and Learning Bruce Lenthall credited the consistent quality of courses to faculty and students. 

“Faculty over the past year have put an extraordinary effort and extraordinary creativity in trying to think about how to make courses work well,” Lenthall said. “Students have put in an extraordinary effort trying to figure out how to learn in this new environment.”

He added, however, that course quality rating was only one specific metric out of many that instructors cared about at Penn. Instructors might also care about whether students learned the material professors intended to convey, Lethal said.

College sophomore Amanda Palamar said her experience taking CIS 160: Mathematical Foundations of Computer Science during summer 2020 was overall positive, although Zoom fatigue made it more difficult to stay engaged with the course material.

“The CIS department and the professor and the TAs did a wonderful job putting together a lot of content,” Palamar said, adding that office hours provided a helpful resource for connecting with TAs.

Course difficulty during COVID-19

Engineering consistently ranked the highest in terms of difficulty, with large spikes during the summer sessions. Difficulty ratings have not exhibited significant changes since the transition to online. 

Nursing courses were ranked the second most difficult, followed by Wharton and the College. 

Palamar said she believed summer courses may be more difficult because of the fast pace required to fit all of the course content into a shorter session.

“Even the professor said that it is very difficult to learn all that content in five weeks,” Palamar said. 

Professor quality ratings

To determine the highest-rated professors, DP Analytics Staff took the average instructor quality rating of all sections that a professor taught within the previous nine semesters. Professors that taught three or fewer sections between spring 2018 and fall 2020 were excluded from the table to prevent outliers from influencing their rating. 

All of the top College professors are from the language and music departments, and the majority of top-rated Wharton professors teach Legal Studies and Business Ethics and Operations, Information, and Decisions. The Engineering professors were distributed across multiple departments, although Computer and Information Science professors make up four of the selected 10. 

French professor Mélanie Péron wrote in an email to The Daily Pennsylvanian that she believes her high course quality ratings are the result of the love she has for her job. 

“I infuse the positive energy I get from the realization of how fortunate I am into my teaching," Péron wrote. "I also truly care about our students and I believe it comes across my teaching and our interactions.”

Péron added that she adapted her course to online learning by reducing busy work, granting more flexible extensions, and using humor in her videos to build rapport with students. 

Course difficulty by school

Course difficulty ratings were analyzed by excluding courses with three or fewer sections. A wide variety of subjects ranked highly in difficulty, including languages, natural sciences, financial analysis, engineering, and nursing clinical. 

Engineering associate professor Swapneel Sheth, who taught CIS 120: Programming Languages and Techniques virtually in fall 2020, said that it was important not to read too much into the difficulty ratings because they could mean different things to different people. Sheth also noted that student surveys suffered from various biases such as female professors getting lower scores than male professors as well as smaller courses getting higher scores than larger introductory courses. 

Course quality and size

As the number of students in a course increases, the course quality rating tends to go down.

Engineering junior Kaiying Guo said that smaller class sizes are more conducive to a welcoming learning environment. 

“In smaller classes, it's easier to be engaged with the material,” Guo said. “It's also less daunting to speak up.”

Critical Writing professor Jacob Friedman, who teaches exclusively seminar courses, agreed. 

“During the pandemic, we've really been deliberate about how to create a sense of community and sense of collaboration in a way that I think smaller classes make possible,” he said. 

Labs and music courses

Lab courses and music courses had to dramatically transform their course content for the online environment. 

Engineering junior Caitlin Frazee, who took MUSC 030: 1000 Years of Musical Listening and MUSC 050: World Musics and Cultures online, said that one of the difficulties faced by music courses was the connection delay on Zoom calls, which made it difficult for students to play in sync. 

Frazee also saw benefits, however, such as the ability for her classmates to share their music with people who do not live in or near Philadelphia.

"I have some friends who have family that live far away and had never heard their kids play in a recital before, so this was the first time they got to see their grandkids perform live," Frazee said. 

Frazee also enrolled in the laboratory courses BE 309: Bioengineering Modeling, Analysis and Design Laboratory I and BE 310: Bioengineering Modeling, Analysis and Design Laboratory II online. She said that one of the difficulties with the labs were that not all members of the lab group were present on campus during the spring semester.

Still, Frazee said the professors and lab managers learned a lot from the experience. 

"They’ve revamped every single experiment so that it can be done remotely," Frazee said. "You can tell that so much time and energy went into making this doable, and you have to commend that.”