Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

US Education Program participants add three times as much quality content as regular new users

Wikipedia Education Program participants from the United States added more than three times as much quality content as regular new users, a quantitative analysis shows.

In the Wikipedia Education Program, professors assign their students to edit Wikipedia articles as a grade for class, assisted by volunteer Wikipedia Ambassadors. In fall 2011, 55 courses participated in the program in the United States, with students editing articles on the English Wikipedia. On average, these students added 1855 bytes of content that stayed on Wikipedia, compared to only 491 for a randomly chosen sample of new users who joined English Wikipedia in September 2011. These numbers establish that students who participate in the Wikipedia Education Program contribute significantly more quality content that stays on Wikipedia than other new users.

Examining the distribution of content that survived on Wikipedia for both of these groups, we found that almost half of the Wikipedia Education Program participants added 1,000 or more bytes that stayed on Wikipedia in the first six months. In contrast, more than half of the random sample of new editors added no content that stayed on Wikipedia in the first six months. The targeted recruitment of students, combined with the support provided by the Ambassador Program and instructors, results in a much larger percentage of new editors who contribute quality content to Wikipedia.

To understand the collective impact of the Wikipedia Education Program in fall 2011, we compared the amount of content students added to Wikipedia to the content added by the random sample of new editors. The numbers show that the 920 student editors who participated in the program in fall 2011 added the same amount of content as 2250 typical new editors (editors are defined as users who made at least one edit to an article). In terms of new content, students have twice the impact as typical new editors.

An important consideration for any outreach project is editor retention. Data showed that students who are introduced to editing Wikipedia through the U.S. Education Program are just as likely to continue editing as any other newcomer.

The Wikipedia Education Program has now grown to Egypt, Brazil and other regions beyond North America. With an increased global presence, measuring and understanding the contributions of new student editors (and how they differ from other new users that join Wikipedia) has gained importance. Establishing a common metric for measuring the impact of the Wikipedia Education Program on various Wikipedias is another key motivation for a quantitative study.

There’s a lot more work to be done on measuring the program’s impact. So, stay tuned for more information about these metrics.

Methodology for this research can be found at: http://meta.wikimedia.org/wiki/Research:Wikipedia_Education_Program_evaluation#Methods

Ayush Khanna, Data Analyst, Global Development

(with input from Mani Pande, Head of Global Development Research)

7 Responses to “US Education Program participants add three times as much quality content as regular new users”

  1. Ayush Khanna says:

    Thanks for the comments.

    @Harry:
    The goal of the program is to improve the quality of Wikipedia articles. That said, we are working towards increasing retention of students after the academic semester ends.

    @James:
    Re 1: A quantitative approach is definitely not the complete picture, but we feel it is a very good approximation. Qualitative research will be more accurate, but also requires a considerably greater amount of effort. That said, we are working with a few community members on defining qualitative metrics for evaluating the program. Please join the discussion at http://en.wikipedia.org/wiki/Wikipedia:Education_noticeboard#Suggested_approach_to_metrics_for_student-edited_articles if you are interested.
    Re 2: See my response to Harry.

  2. A couple of comments:
    1)Just because more bytes of students text stay on Wikipedia than bytes of a random new user does not necessarily mean that they contribute more quality content. It might mean that they are editing less closely watched pages and thus concerning edits are less frequently picked and reverted up by the community.
    *Do not mean to be the devils advocate but the only way to determine if the text is of good quality is to manually go over every change that is made by students and every change that is made by a group of random new users. Per a good study design this should be done by at least two independent people with a third providing input if the two do not agree on the assessment.
    2)Per “An important consideration for any outreach project is editor retention. Data showed that students who are introduced to editing Wikipedia through the U.S. Education Program are just as likely to continue editing as any other newcomer.” We all agree that editor retention of new users is very poor. That this project only matches it is a little disappointing.

  3. Harry Burt says:

    “Data showed that students who are introduced to editing Wikipedia through the U.S. Education Program are just as likely to continue editing as any other newcomer.” Slightly disappointing, I would say :(

Leave a Reply