Why Penn Course Review ratings no longer include written comments from students

Penn Course Review was recently updated with ratings for last semester’s classes — but once again, there are no new written reviews.

A student organization called Penn Labs is responsible for the technological task of updating Penn Course Review every semester after receiving ratings data from the administration. However, they do not include the written comments students make when filling out course reviews.

This is because students’ comments used to be filtered by a separate student organization called Penn Course Review, which is no longer an active club at Penn.

Midway through each semester, Penn Labs receives an email from the University saying that data from the previous semester has been prepared, according to Wharton sophomore and Penn Labs Co-Director Adam Domingoes. Then, the data is securely transferred from the University to Penn Labs.

“We use our own program, which we wrote about five years ago, to cut and slice it and put it into a way that our programs understand. Then we run it through our program and upload it to our database,” he said. “From there, we actually generate every single page on Penn Course Review.”

Domingoes said the process of uploading the data usually takes around two weeks from the time they receive the data. The time it takes to process and upload the data varies depending on the organization of the data. For example, the organization has to manually cross-list some courses.

For classes offered as recently as in 2012, Penn Course Review offers access to a summary of students’ written comments in addition to average numerical ratings. Students continue to provide written commentary in their course reviews, but it is no longer uploaded to the Penn Course Review site.

“It was because the comments were actually manually reviewed. And essentially that component has dropped out — we’ve lost the manpower for it,” Domingoes said. “[Penn Course Review] would go through and filter all the comments. And then after that we would receive that data from them, so we’d know which comments just can’t be uploaded. We just don’t get the data anymore because there’s no one to review them.”

Domingoes said Penn Labs hopes to bring back the written comments, but they must find a way to review written comments without the manpower of a student organization like Penn Course Review. One option is to use Mechanical Turk, a service run by Amazon, to review the comments.

“There’s a lot of politics that go into it, so it’s just an idea and I’m trying to get it to run through the bureaucracy,” he said. “You can essentially get people across the internet to do very small tasks for you. So it would be sending it out in chunks of fifty or a hundred comments, just saying, ‘are these obscene or can they not be posted for other reasons?’”

Mechanical Turk costs a few cents per task, according to Domingoes. However, this alternative means to filter reviews brings up confidentiality concerns. Penn Course Review data is confidential, so reviews would have to be filtered to remove references to Penn, professors and classes in order to ensure anonymity.