The Wikipedia Usability Initiative partnered with Bolt Peters and Davis Research to evaluate the changes we’ve implemented so far and inform our work moving forward.  If you don’t know what changes we are talking about, check out our Beta (including a new skin, new toolbar, improved search, and more) by following these instructions.

Overall, the study confirmed that we are on the right track with our beta features – showing us room for improvements, maybe a bug or two along the way, and work yet to be done!  You can view the full report (and soon the full videos) on our project wiki, but we thought we’d share with you some highlights:

Success

“It was easy, and I wouldn’t have thought it would be that easy.”

“Before there were a lot of tools, and I liked that they were all spread out in front of you, but this actually makes a lot of sense. I had to muddle my way through the older system, but this one seemed fine.”

“Websites don’t have common sense, but programmers do.”

The majority of our 8 interview subjects found and used our features and tools without instruction and with success.  Special victories go to our more spacious and grouped tabbed navigation, improved search and new searchbox location, and built-in toolbar.  In using these features, users were not only less intimidated, but also showed a greater ease of use and increased performance.  All of the 8 users successfully found the “edit” tab with a minimum of hunting; no one resorted to Google to get to the Wikipedia article they were seeking; two of our users even expressed pleasure and delight in the process!  Perhaps small victories, but a major change from our first study if you remember!

Needs Improvement

“Uh-oh, I think I may have made the wrong kind of link before. I’ll go to the preview window to see if this is a link. It would have been nice to just edit it in the preview.”

“This is different, it’s got these hot-links [the table of contents]. That’s nice.”

“Links are so easy to screw up. I’m not sure if we’ve correctly typed the link markup. Ah, there are these buttons…”

Some of our tools are definitely still rough around the edges – their flaws and failures were seen in technicolor when observing people using them.  Our link dialog caused the most confusion.  6 of our 8 users initially made some errors in using it, and some received a false positive assurance when they had not actually accomplished the link behavior they were attempting.  Oops!  Our features need to err on the side of a user’s expectation rather than giving users access to the technical structure or wiki syntax, which they did not in this case.  For example, to create a new link in our prototype, users were asked to specify whether they wanted to create an “external link” (to a website) or “internal” link (to a different article) – a differentiation that exists in wiki code, but not in the eyes of a novice user.  Additionally, our toolbar buttons need to behave consistently and be grouped accordingly.  Having dialogs for links and tables, and not having one for a reference was not acceptable and led to some quite confused and persistent button pushing by our subjects.

Speaking of buttons, our “Bold” and “Italics” toolbar buttons use the roman character “a” – the result of our struggled effort to be accessible to an international community while attempting to take advantage of software standards.  In our effort to generalize, we became too general – even those users who correctly guessed the purpose of these buttons had to hover over or use them to confirm their assumptions.  We’re going all in – look out for our efforts to make our toolbar icons language specific soon!

As if we didn’t already know it – adding media or “embedding a file” was the least understood toolbar action of our study.  Most users avoided it, but when they did the sample text that it inserted provided no additional insight.

Moving Forward

“I’m completely intimidated by that [template].”

“I’m not sure what that is. I’m going to save it and then see, because this preview is too confusing.”

Our study illustrated how large an effect a small change can have and brought to our attention tweaks and enhancements that need to be made to our current features.  It also showed us that we are just a slice of what is a very, very large pie.  We had many deja vu moments seeing users flounder around previewing and saving, many times adopting strange techniques and multiple windows to add a simple sentence.  The terms “code,” “computer lingo,” “html” often came up and highlighted the separation users feel from their content while editing.  The expectation for editing a wiki to be similar to editing a blog or word processing document was still prevalent.  And though our Table of Contents and built-in cheat sheet put out some small fires, when navigating an lengthy article or searching for help, we again heard “there sure is a lot of stuff to read” and “this is where I’d give up.”

As we’ve mentioned before, we cannot tackle the full scope of issues that our study participants surface.  But I think I can speak for our team when I say we all felt a certain amount of satisfaction in the results of those problems we did address and it has only made us more eager to attack new problems and iterate on solutions we’ve proposed.  As always, we look forward to your comments, insights, and feedback!  We also appreciate your contributions during our fundraiser – it’s in part community support like this that makes the Foundation’s work possible.

Parul Vora + the Wikimedia Usability Initiative

Our build/design friends from Because We Can over in Oakland have done some great work for us over the past two years – including some nice entry-way desks, tables, and advice on how to make our humble space look nice.  They’re also an open company that blazes a trail in using open-source software and providing open-source designs. But recently they finished a particularly special, signature production job for us, our brand new Wikipedia globe sign, now hanging in our offices at 149 New Montgomery in San Francisco.

Jeffrey and Jillian have put together a nice blog post that provides a detailed run-through on how they lovingly crafted the sign using their in-shop CNC robot and meticulous hand-painting.  It brings our new space together in an exciting way, and yes – if you walk right up, not only does it glow, but you can help piece together that magnificent globe.

We’ll have more news to share about the Wikipedia puzzle globe in the coming weeks, but for now we’re happy to be able to share the inside scoop on how this lovely sign came together.

Jay Walsh, Communications

OpenMoko (Om), a company that previously created an open source smartphone, has just launched The WikiReader, a dedicated reader device with an offline copy of the entire English Wikipedia (without images) stored on a small chip. With two AAA batteries, the WikiReader will run for several months, as it’s been optimized for low power consumption. The device has a simple LCD touchscreen and three buttons for searching, viewing random pages, and looking up previously viewed pages.

Building such a device is possible because, unlike most information on the web, Wikipedia content is freely licensed, allowing anyone to copy, modify, and re-use it for any purpose, including commercial uses. We’ve played with the device and given feedback during the development phase, but it’s not a Wikimedia Foundation product, and we make no guarantees of any kind for its operation.

The device showcases a great opportunity that free educational content creates: information from Wikipedia and similarly licensed projects can be packed into self-contained devices, including purpose-built ones like the WikiReader, without requiring any kind of Internet connectivity. In other words, it is very much possible to get a copy of the most comprehensive encyclopedia in human history to every person on the planet who would benefit from it.

While this device is targeted at least initially at users in the developed world, the software running on the WikiReader is open source, so that other projects can re-use it in whole or in part. (Information about that will go up on their website soon.) We welcome it as a creative new distribution method for Wikipedia content. Congratulations to Om for launching this product; we wish them the best of luck in the marketplace.

Erik Moeller
Deputy Director, Wikimedia Foundation

One of the wonderful characteristics of Wikimedia’s wikis, including Wikipedia, is that every change ever made to a page is recorded, back to the very first version (compare, for example, the first version of the article about chess with the most recent version of the same article). This characteristic also makes it possible to assign quality assessments to specific versions, thereby giving our readers greater transparency about the perceived current or past quality of an article.

A very powerful software feature called Flagged Revisions makes it possible to systematize such quality assessments.  It’s been in production use in many of our wikis for more than a year now, including the second-largest Wikipedia, the German language edition. Fundamentally it’s a very flexible feature, and different project communities (the German Wikipedia, the English Wikibooks, etc.) can come up with configurations that suit their needs. By means of our public issue tracker, they can then request from the Wikimedia Foundation that such configurations be turned on.

Even though we’ve made no official announcements about this, you may have seen media reports that Flagged Revisions will soon be enabled in the English Wikipedia. Indeed, there is a specific proposal that was developed by the English Wikipedia community, entitled Flagged protection and patrolled revisions. It’s a very thoughtful proposal that attempts to balance the desire for higher quality, and more systematic assessment thereof, with the immediacy of Wikipedia as it exists today, and was supported by a large majority of interested Wikipedia editors. The idea behind this proposal is to allow regular contributors to systematize a first, basic assessment of all edits by new contributors. However, this assessment will be purely for informational purposes to the reader: a reader will see whether or not the version of an article they look at has been patrolled, and if not, whether a prior patrolled version is available.

Only in a small percentage of cases, we would require changes to be patrolled before becoming the default view for readers. The proposal is to do so initially in the case of articles at high risk of vandalism, including high risk biographies of living people, where false information can do the most serious harm to an individual.

A popular media narrative of this proposal (in the cases where it has been reported roughly correctly to begin with) is that it represents a “clamping down” on Wikipedia’s open editing process. That is nonsense. It is presently the case that many high-risk articles are completely uneditable by new contributors, which is referred to as page protection. For example, as a completely new user, you are not able to alter the article about Barack Obama. These kinds of protections of high-risk articles have been common for many years now. If the proposed model works as intended, it will actually allow us to open up many articles for editing which are currently protected from being edited. Edits will have to be patrolled, which is clearly a step up from edits not being possible at all.

It is true that some implementations of Flagged Revisions are more conservative than that. Any edit in the German Wikipedia by a new or unregistered user has to be patrolled before becoming visible to readers. This is definitely not the case in the proposed English Wikipedia configuration. We believe in letting our communities experiment with different approaches in an attempt to find the right balance.

A test wiki for the English Wikipedia configuration has just been set up in the Wikimedia Labs, and we’ll be importing articles from Wikipedia soon and make a broad call for testing. It’s important for us to get this right – we want to make sure that we don’t make Wikipedia harder to use, for our readers or our editors, in the process of deploying this functionality. That said, we hope to be able to deploy Flagged Revisions in production use on the English Wikipedia within 2-3 months.

From Wikimania in lovely Buenos Aires,
Erik Moeller
Deputy Director, Wikimedia Foundation

[UPDATE 8/26] This post originally said that all biographies of living people would be “flagged protected”. This is not correct. The current proposal is for for articles that are currently under normal mechanisms of protection (where new and unregistered users cannot edit) to be eligible for the new protection model, which allows for more open editing. I apologize for the confusion; thanks to Sage Ross for the quick correction.

Beate Eriksen, a Norwegian film-maker and actress, can add another unique claim to her personal history.  Today her newly-minted English Wikipedia article was counted as the three-millionth article created on Wikipedia.  The article was created by user:Lampman at 04:04 UTC, August 17, 2009.  Barnstars (a form of digital recognition bestowed by Wikipedians to Wikipedians) have been flowing in for Lampman since the achievement was announced.

Since its creation the article has already been edited 48 times to include several info boxes, references, and categorization.

English Wikipedia still holds the title for most articles over any other language edition of Wikipedia, but others are seeing impressive growth.  German Wikipedia will shortly push through its first 1,000,000 articles and French won’t be far behind. Currently at just over 13.7 million articles in all languages, we expect to reach 14,000,000 before the end of 2009. Our stats guru Erik Zachte maintains dozens of stats queries in one place that illustrate the growth of projects, trends in editing and participation, and analyses of our traffic.

Congrats to the thousands of Wikipedians who have contributed their time, edit by edit (roughly 326,832,295 since day one), over the past eight years to help English Wikipedia reach this incredible milestone.  Your work has made the web more amazing for hundreds of millions of users around the world. Thank you!

Jay Walsh, Communications

Hopefully by now you’ve had a chance to try the new skin ‘Vector‘ that the Usability Team has implemented as part of our first release, also known as Acai!  If you haven’t yet, we invite you to check it out by going to Appearance > Skin in your preferences and selecting ‘Vector’.  As you play with the new skin and enhanced toolbar, the Usability Team is looking forward to our next release – codename Babaco – the next alphabetical tropical fruit in what we expect to be a delicious series.

We hope to continue adding features to enhance the editing process – including but not limited to further toolbar enhancements; dialogs for the creation of links, tables, and references; and tools to aid in navigation of article content during editing.  Our research, design, and development depends on feedback from users, community members, and interested parties like you.  We invite you to take a look at our work and if you have any opinions, praise, concerns, or criticism, please let us know here.  We look forward to hearing from you!

Parul Vora, Wikimedia Usability Initiative

The open source movement is not only about software and knowledge base creation. There are active movements in user interface design as well. tango! is one of the neatest projects in design collaborative world, contributing in the creation of open source software such as Open Office and Ubuntu. We, the usability team, also benefit from such open source design projects which allow us to reuse their icons by modifying to meet our needs. For example the icon on the right is the new reference tool icon which can be found in the enhanced toolbar. It is the reuse of Gnome Desktop icons from Wikimedia Commons.

The first set of usability enhancements, new tab layout, enhanced toolbar, and reorganized search page, are now available in MediaWiki projects except for right-to-left language wikis such as Arabic and Hebrew. The support for right-to-left languages should be available in a few weeks, so just hang in there. We welcome you to try out the usability enhancements by going into your preferences and enable ‘Vector’ and the enhanced toolbar from Appearance and Editing menus.

I hope you find the new interface easy to interact. Let us know your feedback in the discussion page of the most recent release page.

Naoko Komura
Program Manager, Usability Initiative

Today Wikimedia Foundation staff and volunteers are in the middle of the first-ever Wikipedia Academy in the United States – at the National Institutes of Health campus in Bethesda, Maryland (see Frank’s post for more info).

You can follow the twitter dialog live.  Volunteer/speaker/photographer Mathias Schindler is also uploading photos, which will make their way to the Wikimedia Commons.

The Academy runs through July 16.  We’ll post more findings and discussions from the event as they come together.

Jay Walsh, Communications

Every day millions of people access health information online. We have recently seen some new hard evidence of Wikipedia’s growing prominence as a health information resource. The rapid development and traffic on the English Wikipedia of an article on the 2009 flu pandemic demonstrates this trend.

Since the pandemic broke out in April 2009, Wikipedia has been one of the major sources of information about the topic. Click the “history” tab of the article, and you will see the fascinating story of hundreds of volunteers donating their time and knowledge to make all facts about the swine flu outbreak accessible to millions of readers around the globe. Started on April 24 as a small article of scarcely more than 200 words, the entry in its current version fills more than 21 printed pages and contains a large number of tables, charts, images, citations and references.

And that is just the story of the new article on the pandemic. A quick glance at the page requests for flu and influenza related articles on the English Wikipedia tells a larger story of Wikipedia’s significance for those seeking health-related information online. Whereas these flu-related articles got about sixteen thousand page hits on April 23, this number increased to dizzying 2.86 million page hits only a week later (see chart).

The significance of Wikipedia as a source of online health information has lately been measured by Michaël Laurent and Tim Vickers, both Wikipedia contributors. In an article published by the Journal of the American Medical Informatics Association (“Seeking health information online: does Wikipedia matter”), Laurent and Vickers showed that Wikipedia ranked among the first ten results in 71–85% of search engines.

Now Vickers and Laurent are among the volunteers organizing a Wikipedia Academy with the National Institutes of Health (NIH), the first to be staged in the United States (see our press release). The Academy model was pioneered by Wikimedia Germany in 2006 and has since been replicated by other Wikimedia chapters around the globe. The event series aims at engaging people who are not familiar with wiki culture or communities.

In this one-day event, to be held in Bethesda, Maryland, on July 16th, experienced volunteer Wikipedia editors will talk about Wikimedia’s mission and orient the audience to Wikipedia’s structures and community policies. Medical researchers and other staff members of the NIH will learn how to contribute to Wikipedia’s content and engage with other Wikipedians to further increase Wikipedia’s quality and credibility.

Frank Schulenburg, Head of Public Outreach

Our CTO Brion Vibber offered a fascinating post on the Foundation’s Tech Blog today, highlighting the incredible traffic spike and related problems caused by the news of singer Michael Jackson’s reported death.

Expect the tech blog to be updated as other server developments unfold, and of course the Wikipedia article to go through some fascinating evolution and discussion.

Jay Walsh, Communications