Penn researcher uses Twitter to gauge Common Core debates

Twitter allows anyone with an internet connection and an opinion to have their very own soapbox, and that certainly doesn’t stop when it comes to debates over education and policy.

While Twitter might not be the first place people think of when they hear scientific research, a study from the Consortium for Policy Research in Education released in late February — authored by a Penn professor — looked at the role Twitter plays in debates about the Common Core State Standards Initiative. 

The project, titled “#commoncore: How Social Media is Changing the Politics of Education,” worked with 190,000 tweets over a six-month period between September 2013 and February 2014 that included “#commoncore.”

The Common Core State Standards are part of a curriculum developed in 2009 that has been adopted by 44 states, including Pennsylvania. They have polarized parents and teachers alike. Critics cite the increased emphasis on testing and that the curriculum does not take the needs of non-traditional learners into account. Others praise the Standards for being internationally benchmarked and incorporating recent research about student learning.

Graduate School of Education professor Jonathan Supovitz worked in collaboration with Alan Daly of the University of California, San Diego and Miguel del Fresno of the Universidad Nacional de Educación a Distancia in Madrid, Spain. 

Among the big takeaways was the finding that most of the discussion about the Standards from a sample of tweets was often about broader political issues than the actual standards themselves. The study claims that proponents of the Common Core use more "policyspeak", or logical arguments, while opponents preferred more emotionally evocative statements that the study calls "politicalspeak". 

Summary of the results with interactive graphics are published on an interactive website launched last week. While confidentiality is necessary in typical research studies, since Twitter is publicly accessible data, the researchers were able to make interactive social network webs which link to users' profiles that are included in the study.

“In some ways this does set a new standard for the accessibility of research,” Supovitz said. “I think that Alan, Miguel and I definitely set out to present research in a different way than we were brought up and taught and trained to.”

A part of  the research coded a sample of about 4,500 tweets based on whether they included “policyspeak” or “politicalspeak” as defined by the study. They broke down the sample into groups that the researches observed based on a user’s Twitter actions and interactions: opponents outside of education that care about the Common Core as it relates to other advocacy issues and two groups for those within education that either support or oppose it.

College sophomore Ariel Smith worked on the project over the summer as part of the Penn Undergraduate Research Mentoring program which pairs undergraduates with faculty researchers for summer projects.

She coded many of the tweets used for the study and said it “was a very involved and hands-on process.” The team would categorize the tweets based on several criteria like whether they mentioned a government figure as part of the “policy” versus “political” distinction.

Frederick Hess, who writes the “Straight Up” blog for Education Week, criticized the study on Twitter and on his blog. He argued that the quantity of tweets sampled was too small and questioned how the policy tweets are distinguished from the political tweets.

Supovitz explained the training his research assistants went through before coding the tweets. The two undergraduate research assistants did trial coding, where they would code tweets independently. After, they would compare, and did not start the actual coding until they got to a point where their coding matched up 80 percent of the time.

They would look at what should be considered “policyspeak” or “politicalspeak” or what was an opinion in support or opposition. It took them three times to come into a range of common agreement before beginning the categorization of the entire grouping of tweets.

Hess also took issue with the way that the researchers publicized the 190,000 tweets in their promos of the study when the sample size was only about 4,500.

“I don’t have to ask every citizen in the country to find out what the prevailing opinion is — I just have to be careful about drawing a sample that represents the population,” Supovitz countered. “So I don’t think that that critique holds any water to the conclusions that we drew because we were really careful in our sampling.”

After spending so much time looking at various tweets, Smith reflected on her personal experience working on the project, saying that it made her think more about the quality of information that circulates on the internet. “This project just showed me really how important it is ‘to do your research,’ so to speak,” she said.

Several states are set to take tests associated with the Common Core this spring so the debate has kept its momentum — the researchers report that more recent data shows that the number of tweets and individuals tweeting is even larger than the set in the study. 

“The volume of conversation is actually a little bit larger now than it was then,” Supovitz said. “It’s not an event that receded, it’s an ongoing thing.”

Please note All comments are eligible for publication in The Daily Pennsylvanian.