We’re happy to announce that the Wikimedia Foundation has started testing a new version of the Article Feedback Tool, to engage readers to help improve Wikipedia — and to become editors over time. We’re very excited about this new development, and look forward to getting more people to contribute to Wikipedia as a result.

Earlier this year, a first version of the Article Feedback Tool (“Rate this Page”) was rolled out to all articles on the English Wikipedia.  The idea behind this feature was two-fold: to provide a measurement of article quality from readers and to provide a potential on-ramp for these readers so that some may become editors.  We found through our analysis that while direct quality assessment is a very tricky matter (a rating of the Justin Bieber page says as much about the rater’s opinion of Bieber as it does about the quality of the article), the use of ratings as a form of low-barrier participation showed promise.  We also received plenty of feedback from the community around how we might improve this feature.

In October, we began development of the next generation of the tool (AFTv5).  Instead of focusing on explicit quality ratings, we shifted the direction of the tool towards finding new ways for readers to help build the encyclopedia.  So rather than primarily asking them to rate the quality of the article, we are asking readers for their input on how to improve the article. We are still testing different lightweight quality metrics, as well.

We are approaching this development in several phases.  The first phase, which went live today, is a test deployment of three new versions of the tool on approximately 10,000 randomly selected articles on the English Wikipedia and on a small number of manually selected articles. For examples, see Android, Wikipedia, and Global Warming.

Here is one of the three versions that are being tested:

bucket 11

This new version of the tool asks the reader whether they found what they were looking for, and if not, prompts them to explain what is missing.  The intent of this version is to provide editors with some idea of feedback on what readers are actually hoping to see when they read a Wikipedia article.  This information may then be used by the editing community when deciding how to improve the page.  The other two versions also ask for reader comments, but with different questions: the second version lets you make a suggestion, give praise, report a problem or ask a question; the third version lets you review the article. These new forms were developed by OmniTI, a web development firm, and were based on designs created by the Wikimedia Foundation in collaboration with the Wikipedia community. To learn more, visit the AFTv5 project page.

We are inviting members of the editing community to evaluate the quality of the comments coming in from each of these three versions of the feedback form.  The goal is to determine which of these versions is most effective at providing high quality feedback that can help improve articles.  Aaron Halfaker, a Wikipedia researcher from the University of Minnesota and a WMF contractor, has developed an evaluation tool that will enable Wikipedia editors to systematically evaluate the quality of the feedback provided. Assuming that these new versions provide constructive feedback, the next step would be to expose these comments in Wikipedia.  To that end, a “Feedback Page” is now under development with community input, and will provide a space where editors can view article feedback, moderate the comment stream, and promote the best contributions to the article talk page.

Oliver Keyes, a member of the English Wikipedia community, is under contract with the Wikimedia Foundation as a Community Liaison to involve editors in this project.  In this role, Oliver is moderating discussions, collecting feedback about the tool, and working with the development team to incorporate this feedback.  Many of the ideas that are in the current test versions came from discussions with these editors.  We will continue to work with the community very closely in the next stages of product design and development. If you’re part of the editing community and want to get involved, please email Oliver (okeyes at wikimedia dot org). Our immediate need is to help evaluate the comment streams generated by each option.  Very soon, we will also need editors to help us design the Feedback Page, which will be used to review and potentially act on the feedback comments.

We hope this new feature can help engage a broader community of readers to provide constructive feedback on articles, share what they know and contribute regularly on Wikipedia.

Howie Fung, Senior Product Manager

Fabrice Florin, Product Consultant