Respondents to the December 2011 Editor Survey reported that lack of time and poor interactions with other editors were the biggest obstacles in their way. Editors that primarily worked on more established Wikipedias (for instance English and German) also talked extensively about the abundance of rules and policies.

In an effort to better understand how we can make editing easier for Wikipedians, we asked them two questions in the survey, about the biggest obstacle that prevents them from making a contribution and about one thing they would like to see changed in order to contribute more effectively.

Word cloud of open ended responses to Q26: "What is the single biggest obstacle that you face in making an important contribution to Wikipedia?"

Many respondents spoke of unpleasant interactions with fellow editors, in many cases involving reverts to content they had added. One noted: “if an editor reverts with snotty comments or reverts a major contribution with no detailed explanation, I feel like saying ‘what’s the use?’  so far, I’ve come back after those experiences, but maybe someday I won’t“. Another respondent talked about the problems arising from a few editors assuming ownership of major articles: “… This makes just jumping in and doing significant rewrites feel almost like an act of aggression and one tends to hesitate to do so despite the ability to write well and knowledgeably about the subject.”

When asked what they would like changed, a large part of the respondents talked about a better editing interface: “Simpler editing interface, replacement of markup language with word processor-esque click and go interface – otherwise editing is too time consuming” (a goal the Foundation is pursuing with the development of the Visual Editor). Another noted: “Easier interface. Editing is difficult and if one is not careful, it’s easy to mess the article up.“ Many respondents also mentioned that dealing with complicated expressions (for instance, mathematical equations) and language intricacies (for non-English Wikipedias) is difficult with the current interface. Some also felt that the Wikipedia community needed to do a better job of recognizing people by merit: “Merit should be given to the quality of edits contributed rather than vehement assertions…“, “People can cherry pick or misrepresent sources with impunity, as long as they are polite“.

Word cloud of open ended responses to Q27: "What is the single biggest obstacle that you face in making an important contribution to Wikipedia?"

We plan to use these responses to inform the continuing improvement of Wikipedia.

Ayush Khanna, Data Analyst, Global Development

In December 2011, we conducted an online survey of Wikipedia editors in 17 languages. This is the sixth and final in a series of blog posts summarizing our findings. If you are interested, you can find out more about the methodology of the survey here.

Technical operations (more operations staff, new caching servers, performance metrics, uptime) were chosen as the most important use of donation money by respondents to the December 2011 Editor Survey, followed by the development of technical features for new and experienced editors. On average, respondents felt that 26.7 dollars out of every 100 donated to Wikimedia Foundation should be spent on technical operations, almost twice as much as the next major expense.

a. Technical operations and features for editors seen as most important expense
b. Almost half of the respondents are unaware of Board elections
c. Slight dip in overall performance ratings since April (more…)

As many as 57 percent of Wikipedians who responded to the December 2011 Editor Survey scored a perfect ten on the Wikipedia Editor Satisfaction Index. This is a marked improvement over the 47 percent from April 2011, and is based on a significantly larger sample of 5,911 respondents. The average score improved less markedly (7.76 vs. 7.65 in April), which suggests we have more editors at both extremes.

In an effort to understand the general satisfaction of the Wikipedia community as well as how their interactions and experiences shape it, we had defined the Wikipedia Editor Satisfaction Index (WESI) during the April 2011 Editor Survey. We used responses to two questions on the survey: how they described their fellow editors (picking from a set of adjectives), and whether they believed community feedback had helped them personally. These responses were weighted, and then normalized to a 0-10 rating. We repeated this analysis for the December 2011 survey.

a. More established Wikipedians score higher, though there is a “chasm”
b. Arabic, Italian Wikipedia editors most satisfied; German, Japanese and Portuguese are least satisfied
c. Editors appreciate constructive improvements from others, and the occasional compliment
d. A majority of Wikipedians say the mission of Wikipedia makes them feel their work is important (more…)

As part of the Wikimedia Movement strategic plan, regular surveys among Wikipedia editors are an important way to take the pulse of the community and identify pressing concerns and needs. We are happy to share results from the second editor survey that was conducted in December 2011. We began survey efforts in April 2011, and results from the first survey are available here. We would like to point out that although this blog post and the following ones will be looking at some trends across the April and December survey, 7-8 months is a rather short time to see statistically significant change on important indicators like gender distribution resulting from Wikimedia Foundation initiatives. Here are some demographic data about Wikipedia editors:

a. Wikipedia editors continue to be predominantly men

The gender distribution of Wikipedia editors hasn’t changed since the last survey. Among those surveyed, 90 percent self-identified as males, 9 percent as females and 1 percent as transsexual or transgender. That being said, there was a greater amount of female editors among those respondents who had joined more recently: Among editors who had joined in 2011, 14 percent were female compared to 10 percent for 2010, 9 percent for 2009 and 8 percent for editors who had joined in 2008 and participated in this survey. Possible explanations include that Wikipedia has been attracting a higher ratio of women recently, or that female editors leave the project sooner. There were no significant variations across the major language Wikipedias, with the exception of the Russian Wikipedia, which reported only 6 percent female editors. Also, out of all editors in the US, 15 percent are women, which is significantly higher than any other country of residence. Conversely, there are fewer male editors in US (85 percent) compared to other countries (UK, India, Brazil, Canada) where 90 percent or more of editors are males.  With initiatives like the Teahouse project that engages new editors through outreach, we hope to increase the number of female editors on Wikipedia.

(D15) What is your gender? (n=6503)

D15. What is your gender? n=6503

b. English Wikipedia continues to be the most read and edited Wikipedia

(more…)

Students at universities in the United States and Canada found that contributing to Wikipedia as a class assignment through the Wikipedia Education Program improved their media literacy and technology skills, according to survey results from the fall 2011 term. In the Wikipedia Education Program, professors assign students to contribute to Wikipedia, usually in the form of expanding a stub article, in place of a traditional research paper grade. At the end of the fall 2011 term, we asked students who participated in the U.S. and Canada program to fill out a survey on their experiences. A total of 132 students took the survey, with a little over three-quarters of the respondents from the United States. About 61 percent of the respondents were enrolled in undergraduate courses, while the remainder were enrolled in graduate courses.

Learning outcomes

A series of questions were designed around assessing student learning outcomes. About two-thirds of the respondents agreed that doing a Wikipedia assignment was a beneficial experience, with almost 20 percent of them strongly in favor of a Wikipedia assignment in place of a traditional term paper. Students from the United States and graduate students all reported higher beliefs in the benefits of a Wikipedia assignment. More than half of the respondents felt that doing a Wikipedia assignment improved (1) their ability to identify poor quality Wikipedia articles and (2) their ability to identify bias in documents. In addition, more than half the respondents felt their ability to write a neutral (i.e., balanced point-­of-­view) document improved through a Wikipedia assignment more than through a standard term paper.

These findings indicate that students recognize the media literacy benefits in doing a Wikipedia assignment. As professors have noted, when Wikipedia is not the destination of the student’s research on a topic, but is instead the road, students are forced out of their research comfort zone. Students are required to evaluate the reliability of sources, find journal articles, and write from a neutral point of view to meet Wikipedia’s policy requirements.

Support resources

Student participants use a set of resources when they have questions about editing Wikipedia — online text, Campus Ambassadors, Online Ambassadors, and professors to name a few. Online text is the most commonly used resource, followed by printed materials. Nearly 93 percent of students who consulted their Campus Ambassador found him or her to be helpful, and 74 percent of students who consulted an Online Ambassador said he or she was helpful.

We’ve found that having that support makes a big difference to students. Students can chat with their Campus Ambassadors in person on campus or on wiki, and they can interact with Online Ambassadors on-wiki and through an IRC chatroom where they can get immediate help for quick questions.

Students had a positive interaction with the Wikipedia community of editors when they interacted with them. Students were asked to pick two adjectives to describe their views of the Wikipedia editing community; top responses included “helpful” (72 percent), “collaborative” (39 percent), and “intelligent” (27 percent).

Motivations

We asked students to identify the key motivations for their contributions to Wikipedia. Important factors students reported were getting a grade, interest in their Wikipedia article topic, and the usefulness of their work (i.e., it wasn’t another throwaway assignment). Graduate students reported a broader variety of motivations, when compared to undergraduates. In particular, more than 60 percent of the graduate students gave a high ranking to the fact that their work contributes to a freely accessible knowledge base.

Final comments and looking ahead

Although converting students into longterm editors is not an explicit goal of the Wikipedia Education Program, as many as 46 percent of our respondents expressed interest in continuing to edit Wikipedia.

When students were asked to share the hardest thing about their Wikipedia editing experience, some common themes emerged. Many students mentioned the challenges of learning how Wikipedia works, and how editing an article was a lot more work than they imagined. Almost universally, they talked about how hard it was to learn wiki syntax. The Visual Editor will help alleviate many of these concerns.

To sum up, here’s what one student had to say when asked about any memorable experiences:

“Overall, a great learning experience. Having to really validate anything you say by backing it with a reputable source is incredibly beneficial and students should be exposed to this, especially if they have not had a research methods course in their undergraduate career.”

Ayush Khanna
Data Analyst, Global Development

Wikipedia Education Program participants from the United States added more than three times as much quality content as regular new users, a quantitative analysis shows.

In the Wikipedia Education Program, professors assign their students to edit Wikipedia articles as a grade for class, assisted by volunteer Wikipedia Ambassadors. In fall 2011, 55 courses participated in the program in the United States, with students editing articles on the English Wikipedia. On average, these students added 1855 bytes of content that stayed on Wikipedia, compared to only 491 for a randomly chosen sample of new users who joined English Wikipedia in September 2011. These numbers establish that students who participate in the Wikipedia Education Program contribute significantly more quality content that stays on Wikipedia than other new users.

Examining the distribution of content that survived on Wikipedia for both of these groups, we found that almost half of the Wikipedia Education Program participants added 1,000 or more bytes that stayed on Wikipedia in the first six months. In contrast, more than half of the random sample of new editors added no content that stayed on Wikipedia in the first six months. The targeted recruitment of students, combined with the support provided by the Ambassador Program and instructors, results in a much larger percentage of new editors who contribute quality content to Wikipedia.

To understand the collective impact of the Wikipedia Education Program in fall 2011, we compared the amount of content students added to Wikipedia to the content added by the random sample of new editors. The numbers show that the 920 student editors who participated in the program in fall 2011 added the same amount of content as 2250 typical new editors (editors are defined as users who made at least one edit to an article). In terms of new content, students have twice the impact as typical new editors.

An important consideration for any outreach project is editor retention. Data showed that students who are introduced to editing Wikipedia through the U.S. Education Program are just as likely to continue editing as any other newcomer.

The Wikipedia Education Program has now grown to Egypt, Brazil and other regions beyond North America. With an increased global presence, measuring and understanding the contributions of new student editors (and how they differ from other new users that join Wikipedia) has gained importance. Establishing a common metric for measuring the impact of the Wikipedia Education Program on various Wikipedias is another key motivation for a quantitative study.

There’s a lot more work to be done on measuring the program’s impact. So, stay tuned for more information about these metrics.

Methodology for this research can be found at: http://meta.wikimedia.org/wiki/Research:Wikipedia_Education_Program_evaluation#Methods

Ayush Khanna, Data Analyst, Global Development

(with input from Mani Pande, Head of Global Development Research)

We recently concluded our annual fundraiser, and it was a great success. With over one million individuals pitching in from almost every country, our donors are a diverse and interesting group in themselves. While we don’t directly collect demographic or other information from our donors, we used our readers survey as a means of understanding who our donors are, and what motivates them to donate to Wikipedia. Please note that these results are from a sample of our reader population, not drawn from actual donor data.

a. Only about half of our readers realize that Wikipedia is a non-profit, editors much more likely to donate
b. An appeal from Jimmy is a popular reason for donating, but people in different regions are motivated differently
c. About one-fourth of our respondents said they would donate; readers from US, Egypt and India most likely to do so
d. Readers cite affordability as the biggest reason for not donating
(more…)

In a previous blog post, we discussed our readers’ perception of article quality. In addition, we asked our readers to compare Wikipedia as a whole to other prominent websites – Facebook, Twitter, New York Times, Google, YouTube, Yahoo and CNN. Of course, there are several key differences between them, but we wanted to understand how Wikipedia stacks up against other high-traffic websites.

Readers from all 16 countries in our sample compared Wikipedia’s interface and ease of navigation to other Internet properties. If we look at the sample as whole, Wikipedia (8.09 on 10) was rated a close second to Google (8.44) on these measures. What makes this even more interesting is Wikipedia’s relationship with the search engine, which we mentioned in an earlier blog post. Although ratings varied across countries quite significantly, in most cases there was little deviation in ratings relative to other websites, with some exceptions.

Interface/look and feel

When asked about the Wikipedia interface, readers scored Wikipedia 7.92 out of 10 on average, just behind Google (8.3). About 46 percent of our readers scored the interface 9+ out of 10, compared to 54 percent for Google. We did not find significant deviations across countries or languages, with one exception: Readers in Egypt (and by extension, Arabic speakers) rated Wikipedia lower than YouTube, Facebook and Yahoo. A desire for better right-to-left support is one plausible explanation for the result.

D8a. How appealing do you find the interface or look of the following sites?

Ease of Navigation

Readers scored Wikipedia 8.27 on this metric, slightly lower than Google (8.59). 53 percent of our readers rated the ease of navigation 9+ out of 10, compared to 63 percent for Google. As above, Arabic/Egyptian readers rated Wikipedia below YouTube, Facebook, and Yahoo.

D8b. How easy do you find it to navigate the following sites?

Mani Pande, Head of Global Development Research

Ayush Khanna, Data Analyst, Global Development

We recently conducted an online survey of Wikipedia readers, limited to 250 participants each in 16 countries. This is the seventh in a series of blog posts summarizing our findings. If you are interested, you can find out more about the methodology of the survey here.

In our recently concluded Annual Plan, we identified increasing the number of active contributors as one of our strategic priorities. As of September 2011, there are 79,890 active Wikipedia contributors (active is defined as those making five or more edits in a month), while we want to increase active editors to approximately 95,000 on all Wikimedia projects in June 2012.

a. Only 6% of our readers have ever made an edit to Wikipedia

b. Most readers are happy to just read, many cite lack of expertise

c. Avid Wikipedia readers, readers with heavy online activity, Twitter users, men, younger readers and online contributors are strong candidates for editors (more…)

The Wikipedia Education Program has grown by leaps and bounds since its inception last year, as part of the Public Policy Initiative. In 2011, the program ventured beyond the United States into Canada and India, making the measurements of the program’s impact even more important. We want to use these metrics (some of which are outlined below) as tools that help us understand and improve the Wikipedia Education Program as a whole, while also understanding individual pieces of the system better.

a. Fall 2011 Numbers and Growth

b. Gender Representation

c. Wikipedia Education Program Metrics and Activities Meeting

(more…)