Introducing Wikipedia Editor Satisfaction Index

Translate This Post

The Wikimedia Foundation is working on new products and global initiatives to increase participation in our projects, specifically Wikipedia. To help inform the development of this work we’ve been researching the trends and patterns of Wikipedia editors, most recently through the Wikipedia Summer of Research initiative and also with data from the 2011 Wikipedia Editors Survey.
While studying editor participation trends, we have hypothesized that acrimony and disagreement in the editing community could be a leading cause of a decrease in project participation. To test this hypothesis as a segment of our analysis of responses to the Editor Survey (report here), we defined the Wikipedia Editor Satisfaction Index (WESI). The WESI is a metric gauging the overall satisfaction of the editing community and interactions/assessments of fellow editors. We used responses to two questions on the survey: how they described their fellow editors (picking from a set of adjectives), and whether they believed community feedback had helped them personally. These responses were weighted, and then normalized to a 0-10 rating.
The results were encouraging. About 47 percent of editors surveyed scored 10/10. In all, about 77 percent of those surveyed scored 7.5 or higher, indicating that the majority of our editing community is very satisfied with their experience on Wikipedia and has a healthy assessment of fellow editors. This is great news – as Wikipedia continues to focus on improving the editing experience, while also making efforts to foster new participation (especially in the Global South), the community’s support is vital.
Image (1) WESI-Score-Distribution-April-2011-Editor-Survey.png for post 6203

Distribution of WESI scores across all surveyed Wikipedia editors

In order to understand what factors determine an Editor’s satisfaction with Wikipedia, we performed a multilinear regression1 on the WESI metric. Some interesting findings:

  1. Help is appreciated: Having others from the community add content or correct grammatical mistakes greatly increases the likelihood of an editor reporting a positive experience.
  2. Peer recognition matters a lot more than any other kind of recognition: Editors highly value the respect and recognition of their peers. Editors who received barnstars or any other form of reward from their peers were much more likely to report a higher score. Interestingly, events like having an article featured or promoted to the front page did not have a very significant effect on editor satisfaction.
  3. Explanations for reverts are key: When an edit is reverted, not explaining why has a strong negative impact on editor satisfaction. Similarly, an explanation actually has a strong positive influence on editor satisfaction.

A comparision of WESI scores reveals that women are, on average, less satisfied than men, though not by much – about 5 percent. Although transsexuals/transgenders (marked below as Others) together account for only 0.5 percent of our sample, it’s important to note that their satisfaction scores are significantly lower.

Image (2) WESI-Comparison-By-Gender-April-2011-Editor-Survey.png for post 6203

WESI score comparision by Gender

The Editor Survey Report highlights some more findings, but the emerging theme is simple: be nice to each other, and help out where you can!
As we work towards establishing the WESI metric as a standard for understanding the community’s experiences on Wikipedia, we’ll continue to share more findings (and implications) of the Wikipedia Editor Satisfaction Index.
Mani Pande, Head of Global Development Research
Ayush Khanna, Global Development Intern
(This is the eleventh in series of blog posts where we previously shared insights from the April 2011 Editors Survey.)
1: http://en.wikipedia.org/wiki/Regression_analysis

Archive notice: This is an archived post from blog.wikimedia.org, which operated under different editorial and content guidelines than Diff.

Can you help us translate this article?

In order for this article to reach as many people as possible we would like your help. Can you translate this article to get the message out?

3 Comments
Inline Feedbacks
View all comments

Removing the default edit summary text for “undo” and similar actions would help a great deal. Unexplained reverts are poison to collaboration, and most editors wouldn’t leave the edit summary blank even though they won’t modify default undo summary text. Use tags from the filter rules instead of wasting the edit summary.

How did you select the members of your sample? I hope you chose editors of a variety of tenure lengths. Otherwise, your results will be skewed toward the positive side (and it will probably be skewed toward the positive even if you did choose a variety of tenure lengths); those who are dissatisfied with Wikipedia tend to become inactive or leave, so they might not receive an invitation to your survey, and their experience may be overlooked. Did you contact any editors who have been inactive or left Wikipedia? You might also investigate the unyielding attachment to the status quo.… Read more »

Dear J, according to the report, the methodology is described thus: “The survey was conducted online on all Wikipedia language sites in the last week of April. Using the help of the community, the survey was translated into 21 languages beyond English, including: Chinese (traditional, Hong Kong), Chinese (simplified), Serbian, Russian, Portuguese, Polish, Dutch, Macedonian, Italian, Hungarian, Croatian, Hebrew, French, Finnish, Spanish, German, Danish, Welsh, Catalan, Bulgarian and Arabic. The survey was conducted in native languages for which translations were available, and for the remainder of Wikipedia language projects the survey was available in English. The survey was limited to… Read more »