Students in the Wikipedia Education Program in the United States and Canada improved article quality on the English Wikipedia by an average of 88 percent during Spring 2012, according to new research conducted by Luis Campos, an external data analyst. In the Wikipedia Education Program, professors assign their students to improve course-related articles, with support from Wikipedia Ambassadors who help students learn the basics of Wikipedia editing.

Experienced Wikipedia editors evaluated a random sample of articles students worked on as part of the Wikipedia Education Program in the United States and Canada in Spring 2012. The metric evaluators used, with assessment areas for comprehensiveness, sourcing, neutrality, readability, formatting, and illustrations, on a 26-point scale, is based on the Wikipedia 1.0 metric used across English Wikipedia. Evaluators provided two ratings, one for the article quality immediately prior to the first edit the student made, and one after the class had wrapped up their work; reviewers also used the same metric to evaluate articles that students created from scratch. A total of 124 articles formed the sample. Altogether (counting both new and pre-existing articles), articles improved on average 6.5 points, from 7.4 to 13.98 points on the 26-point scale. The graph below shows the quality distribution of articles before students worked on them (in blue), and the quality distributions of articles after students worked on them (in red).

Article quality improvement of sample of Wikipedia articles edited by students participating in the Spring 2012 Wikipedia Education Program in the United States and Canada. This graph shows overall improvement (both existing and new articles).

The 124-article sample included 82 existing articles and 42 new articles created by students. Existing articles improved 2.94 points on average, from 11.26 to 14.2, with the most improved article improving by 10.25 points. An example of such an article that a student improved is the article on vocabulary development. You can see the versions prior to students’ first edits and the status it was after the class finished. The graph below shows the distribution of pre-existing articles before (blue) and after (red) student work.

Article quality improvement of sample of Wikipedia articles edited by students participating in the Spring 2012 Wikipedia Education Program in the United States and Canada. This graph shows improvement of existing articles only.

New articles had an average score of 13.55. You can see a sample of what a student contributed to a new article by reading Temptation, a Václav Havel play. The graph below shows the distribution of quality of new articles students created through the Wikipedia Education Program.

The Spring 2012 numbers show improvement over the 2010–11 quality of students contributions from the Public Policy Initiative pilot of the U.S. program, where articles improved an average of 5.8 points. We’re encouraged to see improvement in Wikipedia’s article quality through the Wikipedia Education Program.

LiAnna Davis, Wikipedia Education Program Communications Manager

Tina Loo, a history professor at the University of British Columbia, recently incorporated Wikipedia in her North American environmental history class (HIST 396), the first history class in the Wikimedia Foundation’s Canada Education Program, which promotes editing and improving Wikipedia articles as part of educational curricula. The 60 students in her class worked in teams to expand existing Wikipedia articles and start new ones. In total, they worked on thirteen articles, all based on subjects in Canadian environmental history.

Though Loo said that she was “attracted by Wikipedia co-founder Jimmy Wales’ philosophy of making knowledge accessible (at least to those who have an internet connection),” she had a more personal reason to participate in the program.

For the past two years, a pile of unclaimed HIST 396 term papers has accumulated in the corner of my office, evidence of my failure to engage students adequately. It was as if the energy and anxiety that went into these fifteen- to eighteen-page tomes dissipated completely when they were handed in. The authors of these abandoned papers didn’t, it seems, care how their ideas and arguments resonated with their audience: me. Increasingly, it seemed ridiculous to have students spend time doing something they weren’t interested in and for me to spend time doing something they weren’t interested in, namely writing comments.

As part of the Wikipedia Education Program, Loo took advantage of the resources developed and provided by the Foundation. She also availed herself of several Campus Ambassadors who were graduate students at UBC. They volunteered to help her as she taught her students how to edit Wikipedia and how to navigate the principles and practices that guide the Wikipedia community of editors. In addition, three Online Ambassadors – “The Interior,” “Wetman,” and “maclean” – assisted Loo from a distance, a fact that struck her as remarkable.

They’re members of the Wikipedia community, expert editors who volunteered to help my students and me with writing articles because they just happened to be interested in the subject matter. Beyond their handles and what each said about themselves on their Wikipedia “Talk” pages, I have no idea who they are. Their names materialized on my Wikipedia assignment syllabus within hours of me putting it up.

For Loo’s students, the experience was novel, and it compelled them to consider that the product of their studies would be seen by more than just their teacher, or their parents.

The public nature of Wikipedia, and the fact students felt they were contributing to something that would live on after the class was over made the task of writing an entry exhilarating and intimidating at the same time. The self-consciousness that came from writing something that wasn’t just for me, a TA, or, at most, the other students in the class, translated into a level of care about both the form and content of their writing that I don’t always see.

The students soon discovered that it wasn’t just the prospect that their “term papers” would be read by thousands or tens of thousands of people around the world, but they would also scrutinized by Wikipedians, who are exacting in their standards.

When Wikipedians like these took issue with what they wrote, the students couldn’t just be self-conscious: they also had to respond. Learning how to explain why they had written what they had, to defend it respectfully, and to modify it in light of valid criticism was incredibly valuable. I was impressed with how the students stood up for themselves, especially given that not all community members abided by the first rule of Wikipedia: “Don’t bite the newbies!”

Though Loo detailed some of the less civil comments, she also found a number of Wikipedia editors came to the defense of her students and encouraged them to improve their work. She said that it would be impossible to replicate the kinds and degree of questions that her students received from Wikipedia editors, giving them a remarkable experience with peer review. It also gave them a better insight into how knowledge is developed.

In the end, the value of the Wikipedia assignment lies in giving students first hand experience in constructing knowledge. Writing the articles showed them how it’s made; that it changes over time, and it does so in part as a result of competition and cooperation. Knowledge is a compromise, willing and grudging. It’s the outcome of exercising power and it is powerful.

Tina Loo is a History Professor at the University of British Columbia. Read the extended account of her experience on Niche. Readers interested in the particulars of Loo’s class can find more information at the Wikipedia assignment syllabus for HIST 396.

Students at universities in the United States and Canada found that contributing to Wikipedia as a class assignment through the Wikipedia Education Program improved their media literacy and technology skills, according to survey results from the fall 2011 term. In the Wikipedia Education Program, professors assign students to contribute to Wikipedia, usually in the form of expanding a stub article, in place of a traditional research paper grade. At the end of the fall 2011 term, we asked students who participated in the U.S. and Canada program to fill out a survey on their experiences. A total of 132 students took the survey, with a little over three-quarters of the respondents from the United States. About 61 percent of the respondents were enrolled in undergraduate courses, while the remainder were enrolled in graduate courses.

Learning outcomes

A series of questions were designed around assessing student learning outcomes. About two-thirds of the respondents agreed that doing a Wikipedia assignment was a beneficial experience, with almost 20 percent of them strongly in favor of a Wikipedia assignment in place of a traditional term paper. Students from the United States and graduate students all reported higher beliefs in the benefits of a Wikipedia assignment. More than half of the respondents felt that doing a Wikipedia assignment improved (1) their ability to identify poor quality Wikipedia articles and (2) their ability to identify bias in documents. In addition, more than half the respondents felt their ability to write a neutral (i.e., balanced point-­of-­view) document improved through a Wikipedia assignment more than through a standard term paper.

These findings indicate that students recognize the media literacy benefits in doing a Wikipedia assignment. As professors have noted, when Wikipedia is not the destination of the student’s research on a topic, but is instead the road, students are forced out of their research comfort zone. Students are required to evaluate the reliability of sources, find journal articles, and write from a neutral point of view to meet Wikipedia’s policy requirements.

Support resources

Student participants use a set of resources when they have questions about editing Wikipedia — online text, Campus Ambassadors, Online Ambassadors, and professors to name a few. Online text is the most commonly used resource, followed by printed materials. Nearly 93 percent of students who consulted their Campus Ambassador found him or her to be helpful, and 74 percent of students who consulted an Online Ambassador said he or she was helpful.

We’ve found that having that support makes a big difference to students. Students can chat with their Campus Ambassadors in person on campus or on wiki, and they can interact with Online Ambassadors on-wiki and through an IRC chatroom where they can get immediate help for quick questions.

Students had a positive interaction with the Wikipedia community of editors when they interacted with them. Students were asked to pick two adjectives to describe their views of the Wikipedia editing community; top responses included “helpful” (72 percent), “collaborative” (39 percent), and “intelligent” (27 percent).

Motivations

We asked students to identify the key motivations for their contributions to Wikipedia. Important factors students reported were getting a grade, interest in their Wikipedia article topic, and the usefulness of their work (i.e., it wasn’t another throwaway assignment). Graduate students reported a broader variety of motivations, when compared to undergraduates. In particular, more than 60 percent of the graduate students gave a high ranking to the fact that their work contributes to a freely accessible knowledge base.

Final comments and looking ahead

Although converting students into longterm editors is not an explicit goal of the Wikipedia Education Program, as many as 46 percent of our respondents expressed interest in continuing to edit Wikipedia.

When students were asked to share the hardest thing about their Wikipedia editing experience, some common themes emerged. Many students mentioned the challenges of learning how Wikipedia works, and how editing an article was a lot more work than they imagined. Almost universally, they talked about how hard it was to learn wiki syntax. The Visual Editor will help alleviate many of these concerns.

To sum up, here’s what one student had to say when asked about any memorable experiences:

“Overall, a great learning experience. Having to really validate anything you say by backing it with a reputable source is incredibly beneficial and students should be exposed to this, especially if they have not had a research methods course in their undergraduate career.”

Ayush Khanna
Data Analyst, Global Development