There has been a lot of interest lately in Flagged Revisions, a quality control mechanism for MediaWiki. In particular, people want to know when and how that’s getting used on the English language Wikipedia.
I’m William Pietri, a San Francisco software consultant who recently came on part time to do project management for this. In addition to my long experience building web software for on-line communities, I’m also a Wikipedian. Although I haven’t done much more than small corrections lately, I started editing in 2004, and became an admin in 2007.
My job on this has two main parts. The first is to make sure that everybody working on this gets everything they need to make progress. The second is to communicate progress to the wider world. In the spirit of open communication, this is an update in question-and-answer form. Mostly from real questions people have actually asked me, but I’m going to sneak in a couple that I expect somebody will ask shortly.
What is Flagged Revisions?
You can find more detail here, but Flagged Revisions is basically a way to insert a quality review step between someone editing an article and that article version being published for the general public to see. It has been in use on the German Wikipedia since May 2008, and implemented in other languages and projects since then (see this page on Meta for a full list). Typically, in those use cases, every single article is treated in this way, and every change by a new user has to be reviewed. Â There are a number of ways Flagged Revs can be used, and the proposed implementation for English Wikipedia (described below) is quite different.
Fundamentally, the objective of this technology is to reduce the exposure of readers both to subtle and not-so-subtle malicious changes in articles (whether it’s the insertion of blatant nonsense, or claiming the death of a celebrity), and to reduce the workload of people patrolling these changes by reducing duplication of effort.
What about the English language Wikipedia?
The use of Flagged Revisions on the English Wikipedia has been under discussion for a long time. Ultimately, the English Wikipedia volunteer community developed a proposal titled “Flagged protection and patrolled revisions“, which garnered strong support. It is fundamentally different from the way the technology has been used so far. Instead of requiring every change by a new or untrusted user to be reviewed, the mechanism would be activated on a per-page basis only, as an alternative to existing tools to restrict editing.
Notably, thousands of articles in the English Wikipedia, typically pages with a very high risk of malicious editing (e.g. major political figures), are currently “semi-protected”, meaning that new or unregistered users cannot make any changes at all. This new tool would make it possible to open up these pages for editing, provided that potentially problematic changes receive positive review. As a result of the more open approach, more high-risk pages could be made subject to this level of community moderation.
Initially, the English Wikipedia volunteer community wants to trial the system for two months. In addition to this alternative to page protection, the proposal calls for implementation of a new feature called “patrolled revisions”, which doesn’t impact what readers see, but is designed to make it easier for change patrollers to organize their work.
How is the Wikimedia Foundation responding to this proposal?
The Wikimedia Foundation (WMF), together with Wikimedia Germany, has driven and funded the development of the FlaggedRevisions technology since 2007 to the point where it has been able to scale to more than a year of production use in our second-largest project, the German language Wikipedia. WMF has carefully reviewed the English Wikipedia proposal, and allocated resources to assess its impact and support implementation along the principles outlined in the proposal.
The technology as proposed markedly differs from the way it’s been used before, so it’s a substantial development and design effort to get it right. For example, the notion of a per-page setting necessitates an entire set of user interface changes that allow changing the setting of a page, and that make it clear to a reader what the state of a particular page is. See this mock-up by Howie Fung as an example of what revised per-page controls could look like. WMF will post further mock-ups for feedback and prototypes for testing as they are built.
Who is currently working on the project?
- Aaron Schulz, a contract developer with Wikimedia, is the lead developer of FlaggedRevisions.
- Howie Fung, a contract product manager who also works with Wikimedia’s Usability Initiative, is supporting usability and product review of the software.
- I, William Pietri, support the project management of the English Wikipedia roll-out as described above.
- Erik Zachte, Wikimedia’s Data Analyst, will develop metrics specifically assessing the impact of the English Wikipedia rollout.
When will it be done?
This question has been asked a lot lately. Because this isn’t the flipping of a switch but a software development project, answering it requires me to let you in on a secret about software development projects. There are basically four ways to deal with dates, but only three of them are sane:
- It’s done when it’s done. Nobody mentions dates. The developers code until they’re finished. Then you release, get feedback, and code some more.
- Measure progress and project dates. You lay out all the work, estimate relative size, and then measure how much you get done over time. That data is used to figure out release dates.
- Pick a date and release whatever you finish. If you’re building, say, annual tax return software, it’s better to ship on time and drop features than it is to finish late with everything.
- Make up dates to please people. This is very popular, and has the advantage of making people happy at first, but it rarely works out well.
Until recently, we were using the first approach. That’s how most Mediawiki development (and most other open source development) works, and it has many advantages. But because a lot of people are eager for this project to launch, we’re shifting to the second approach.
The developer, Aaron Schulz, has estimated all of the items on the work list and already started in on them. The holidays complicate things some, but I expect we’ll have enough data to make a first guess at the estimated release date by the middle of January.
Wait, there’s only one developer on this? Is the Wikimedia Foundation taking this seriously?
Yes, absolutely. Aaron has been working on the Flagged Revisions extension for years, and nobody knows it better. We talked about adding developers, but unfortunately adding more people now wouldn’t help. I haven’t dug into the history much, but it looks like the real slowdown lately wasn’t in the coding; it was in turning the many-voiced community response into a clear set of things to do.
Having spent time with all the people involved, it’s clear to me that the Foundation takes this project very seriously. It’s one of small number of high-priority projects, which include things like keeping the site running, organizing the annual fundraiser, and Wikimedia’s usability initiative.
How can I keep track?
There are a few ways. First, we’ll mention big updates (and the eventual release) here on this blog. Second, keep an eye on the labs site. That will be updated regularly with the latest code and configs. You can judge for yourself how we’re doing, and make sure we do it right. And third, I’ve put the work queue into a public web-based tool called Pivotal Tracker. It’s one of the few software project management tools made for the measure-and-project approach we’re using; if you’re the sort of person who likes way too much detail, you can find real-time updates there.
How can I get involved?
Go to the labs site, play with the current implementation, give feedback, post your own user interface design suggestions, report bugs, and so forth. Further community discussion in the English Wikipedia about the proposed roll-out is happening on the page about flagged protection and patrolled revisions. I’m always glad to hear feedback, either on my talk page or via email. And of course, you can comment on this very blog post.
William Pietri
Contractor, Wikimedia Foundation
Can you help us translate this article?
In order for this article to reach as many people as possible we would like your help. Can you translate this article to get the message out?
Start translation