Major news in September include:

• The deployment of protocol-relative URLs, paving the way for native HTTPS;
• Major progress by our new Internationalization & localization team;
• The deployment of our new mobile front-end, now the default mobile experience;
• The start of the deployment of MediaWiki 1.18 to Wikimedia sites;
• The completion of the Google Summer of Code project;
• Ongoing preparation for the upcoming coding events in the USA, the UK, India and online;
• A revamped, and now maintained, roadmap for Wikimedia engineering efforts.

(more…)

With contributions by: Tbayer, Daniel Mietchen, DarTar and Jodi.a.schneider

(more…)

As an increasing number of people access the internet from their mobile phones Wikipedia needs to become increasingly mobile. Recently we wrote about the new mobile frontend but how do you get to a Wikipedia article in the first place, especially if you don’t know what you’re looking for or don’t speak the local language?

Introducing QRpedia.
QR codes – barcodes for the internet – have been around for decades and the technology is increasingly being used in everything from street advertising to museum object labels. QRpedia takes the concept one step further to allow a single QR code to send you seamlessly to the mobile-friendly version of any Wikipedia article in your own language. This system is unique to Wikipedia because no other website has manually created links between languages across such an incredible breadth of topics.

A QRpedia code for the Wikipedia article about the artist Joan Miró. 1 code, 40 languages. Try this one for yourself!

When you scan the code the language setting of your phone is also transmitted. QRpedia uses Wikipedia’s API to determine whether there is a version of the chosen Wikipedia article in the language your phone is using, and if so, displays the mobile-friendly version. If there is no article (yet!) in your preferred language it will show you the most relevant article instead.

Launched in April this year, the open source QRpedia was developed out of the partnership between the Derby Museum and Gallery, England and local Wikimedia contributors Roger Bamkin, chair of Wikimedia UK, and Terence Eden, a mobile web consultant. As “Wikipedian in Residence” at the Derby Museum, Roger capitalised on this system by hosting the hugely successful Multilingual Challenge (map of participants) to ensure that content of key importance to the museum was translated into as many languages as possible. Terence built the system and the museum was kind enough to install object labels incorporating the codes.

In an era when cultural funding is very constrained, the combination of QRpedia and the global Wikipedia community enabled the Derby museum to produce a multilingual visitor experience at virtually no cost. Easy mobile access to Wikipedia articles allows visitors to the museum to access unprecedented detail about the objects and their context – information that didn’t make it onto the exhibit label.

Jimmy Wales scanning the QRpedia code at the working antique carousel in the Children’s Museum of Indianapolis.

This system is now in use in other museums around the world. These include exhibitions at the on-site museum of the the National Archives of the UK, in the permanent signage of key objects at The Children’s Museum of Indianapolis and in a major traveling exhibition of Miró’s work in association with the Fundació Joan Miró of Barcelona.

To generate your own QRpedia codes visit http://qrpedia.org/
and simply paste the URL of any Wikipedia article into the box.
The freely licensed sourcecode can be viewed at http://code.google.com/p/qrwp/

—-

Liam Wyatt
Cultural Partnerships Fellow

In July we enabled protocol-relative URLs on testwiki, and asked for bug reports. We did this in preparation for native HTTPS support for the sites. We received and fixed a number of protocol-relative related bugs, and then tested on a few of the larger wikis. We are now at a point where protocol-relative URL support is stable enough to enable it on all wikis, so today we’ve enabled it.

For information about what protocol-relative URLs are, why they are needed, and how it’ll affect you, see the post written in July. In brief: this changes most links we output in our content from looking like http://www.example.com to //www.example.com . The change shouldn’t affect you.

If you find any bugs related to protocol-relative URLs, please submit a bug report. Known issues are linked from the tracking bug.

Ryan Lane
Operations Engineer

EDIT Sep 28 14:29 UTC: Because of reported breakage in iOS clients, the API’s action=parse interface has been hacked not to return protocol-relative URLs. This is a temporary hack that you should not rely on; fix your clients instead. For details, see http://lists.wikimedia.org/pipermail/mediawiki-api-announce/2011-September/000024.html

Community Fellow, Jon Harald Søby

Our fellowships program is growing, and I’m pleased to announce Jon Harald Søby as our newest Community Fellow.  Jon brings 6 years of experience in the Wikimedia community to his fellowship: he is an active editor, translator and vandal fighter for several Wikimedia projects, an admin and bureaucrat for Norwegian Wikipedia, and in the past has also been a steward, board member for Wikimedia Norge, member of the election committee for Wikimedia Norge and Wikimedia Sverige, and OTRS volunteer.

Language and localization are a major area of interest for Jon. Not only does he help translate the MediaWiki interface and extensions, he’s also a founding member of the Language Committee and is finishing up a BA degree in Linguistics from Norwegian University of Science and Technology. He has traveled widely and is always thinking from a global perspective. All of this makes Jon a great fit for his fellowship project, which is focused on piloting a new model for volunteer translations in the 2011 Fundraiser. (Full disclosure: Jon started working with WMF earlier this summer as a Fundraiser Production Coordinator, but we just had to make him a Community Fellow when we saw how much potential there was for this project!).

Jon’s fellowship will run until February 2012. His project priorities include recruiting and coordinating more translators for more languages, building pages and processes that make it easier for new volunteers to get started, and improving systems for producing high-quality translations. To learn more about the project or volunteer to help, please visit the 2011 Fundraiser Translations page and sign up to be a translator.

Siko Bouterse
Head of Community Fellowships

At the same time as the #WCI11 or the Wikiconference India, there will be a genuine MediaWiki hackathon. The focus of this event will be to crush the technical obstacles that prevent Wikipedia and its sister projects to thrive in India.

This hackathon will be the first held in Asia. Many seasoned developers will be coming to Mumbai to learn first hand what can be done and see what can be done there and then.

To make it a success, there is a Wiki page with our current ideas for the hackathon. The premisses will have rooms for break-out sessions, there will be plenty space, power, internet connectivity, coffee, tea and munchies.

Most important will be that we will be there to learn from you and to show you what we are working on. The Localisation team will be there, the off-line people will be there and the mobile team will be there. We need to meet the many people working on Open Source in India because we want to make sure that whatever we will do fits in with what is already there.

We hope to see you in Mumbai.

Thanks,

Gerard Meijssen
Internationalization / Localization outreach consultant

Congratulations to the seven Google Summer of Code students who made it through the summer of 2011! They all accomplished a great deal, but want to continue contributing to ensure their work maximally benefits Wikimedia.

MediaWiki participated in Google Summer of Code 2011.

Yuvi Panda‘s assessment parsing/aggregating extension aims “to make it easier to select and export article selections for various offline collections.” Yuvi needs some code review and suggestions on how to improve it to meet the Foundation’s quality standards for deployability, as he wrote the developers’ mailing list.

Salvatore Ingala worked on making gadgets customizable. As he elaborated, that means:

• “allowing gadgets to easily declare the list of configuration
  variables they have;
• allowing users to easily change those settings, with an easy-to-use
  UI integrated to the Special:Preferences page.”

The next step is merging his code into trunk, which Salvatore’s planning with other MediaWiki developers.

Kevin Brown created the ArchiveLinks project to address the problem of linkrot on Wikipedia:

In articles we often cite or link to external URLs, but anything could happen to content on other sites — if they move, change, or simply vanish, the value of the citation is lost. ArchiveLinks rewrites external links in Wikipedia articles, so there is a ‘[cached]‘ link immediately afterwards which points to the web archiving service of your choice. This can even preserve the exact time that the link was added, so for sites which archive multiple versions of content (such as the Internet Archive) it will even link to a copy of the page that was made around the time the article was written.

Kevin’s next step: getting a security review of his code, getting a starter feed set up so that the Internet Archive can start archiving it, and campaigning to interest Wikimedians and thus eventually get consensus to turn it on. At least one Wikimedian has already praised Kevin for his work.

Akshay Agarwal wrote a MediaWiki extension, SignupAPI, that makes it easier for a new user to create an account. “This extension creates a special page that cleans up SpecialUserLogin from signup related stuff, adds an API for signup, adds sourcetracking for account creation & provides Ajax-ified validation for signup form.” Akshay’s waiting for code review and discussion before the project can move forward further and benefit Wikimedia users.

Seven students contributed to various parts of MediaWiki, the wiki software that supports WMF sites.

Yuvi, Salvatore, Kevin, and Akshay all worked on features that they aim to get into Wikimedia Foundation-run wikis, such as Wikipedia, Wikisource, Wikinews, etc., sooner rather than later. In contrast, three students worked on extensions that will primarily benefit the larger MediaWiki community. For example, Yevhenii Vlasenko‘s project was a “UserStatus” feature for SocialProfile. The SocialProfile extension is not currently deployed on any WMF wikis, but will benefit several other MediaWiki administrators and users. Zhenya finished his work but would like to continue by integrating better with social networks.

And two students worked on Semantic MediaWiki, which is also not currently deployed on any Wikimedia Foundation sites. Devayon Das made a “QueryCreator” and other improvements, and hopes to simplify its layout, make its interface easier to use, and add some features. And Ankit Garg worked on “Semantic Schemas”.

Congratulations to the students and their mentors.  Here’s hoping they’re all here to help out when next year’s interns roll in! :-)  And I’m looking forward to meeting Kevin and Salvatore, and introducing them to other Wikimedia & MediaWiki developers, at the New Orleans developers’ meetup next month.

Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

Identifying language abilities has been real popular. The “Babel templates” are quite popular on the English Wikipedia and, many of the literally hundreds of templates have been copied to other wikis.

With a limited knowledge of a language, people can be really effective execute many tasks. It helps however when they are addressed in an understandable way.

At translatewiki.net we have been using the Babel extension for a long time; it does not use any templates and all the languages used in any of the WMF projects are supported. As it has been in use for so long, it became really rich in localisations.

Using the Babel extension is easy; my Babel user information for instance can be seen to the right and the syntax for this box can be seen below.

Well, wonder no longer! To configure the Wikimedia servers, we use Puppet, a configuration management system, which lets us write code that manages all of our servers like a single large application. Of course, to really know how our servers are configured, you’d need to see our Puppet configuration.

Good news: we’ve just released our Puppet configuration in a public Git repository.

What is and isn’t included

Basically everything is included in the repository. We spent a few weeks removing private and sensitive things from the repository, though. We have these in a private repository that is only available to Wikimedia staff and volunteers with root access.

This, of course, means that the puppet configuration, as released, won’t completely work. The public repository makes references to files and manifests in the private repository. To make the repository work, you’ll need to fill in the missing information. There isn’t very much in the private repository, though, so that task should be fairly easy.

The point of making this repository public

We have a couple reasons for making this repository public:

1. It shares knowledge with the world
2. It lets us treat operations like a software development project

Both reasons align with our mission, but we were already mostly sharing this knowledge via wikitech. The second reason aligns more closely with our mission, as it allows us to let the world be directly involved in our operations efforts.

Labs and community oriented operations

The release of this Puppet repository is the first step in the Wikimedia Test/Dev Labs project. We’ll be going further than just making the repository readable by the world. Part of the Test/Dev Labs project is to create a clone of our production cluster. This clone will run a branch of the puppet repository.

Staff and community developers, and staff and community operations engineers will be able to push changes to the test branch of the Puppet repository, which will manage the cloned cluster. They’ll then be able to push these changes for review to the production branch of the Puppet repository. The staff operations engineers can then code-review the changes and push the changes out to the production systems.

Like the Wikimedia content, the site interface, and the site’s software (MediaWiki), community members will be able to edit the site’s architecture as well.

Accessing the repository

Since this is a public Git repository, you can do an anonymous git clone like so:

git clone https://gerrit.wikimedia.org/r/p/operations/puppet

You can browse the repository through the gitweb interface. You can see the code review activity via Gerrit.

Ryan Lane
Operations Engineer

[Update 2011-09-24: The initial test deployment and stage 1 have gone well, with only minor glitches that we've mostly cleaned up.  Stage 2 and 3 are currently on schedule.  We've decided to add incubator.wikimedia.org to the list of wikis we'll be deploying to, which is reflected below.]

MediaWiki 1.18 will soon be deployed to all Wikimedia sites, including Wikipedia. As you may know, MediaWiki is the wiki software developed by the Wikimedia community, and 1.18 is the upcoming version of the software that has been in development since December.

Thanks to the completion of the heterogeneous deployment project, we are now able to run different versions of MediaWiki concurrently on Wikimedia sites. This means that we don’t have to upgrade all sites at the same time any more, which should limit the problems we encounter.

The deployment is scheduled to happen in several stages, starting next week:

• Monday, September 19, 23:00-01:00 UTC — Production test: test2.wikipedia.org – this stage will ensure that 1.18 is compatible with the rest of our production infrastructure. There’s a small chance that changes here could affect all wikis.
• Wednesday, September 21, 23:00-03:00 UTC — Stage 1: simple.wikipedia.org, simple.wiktionary.org, usability.wikimedia.org, strategy.wikimedia.org, mediawiki.org, he.wikisource.org
• Monday, September 26, 23:00-03:00 UTC — Stage 2: meta.wikimedia.org, en.wikiquote.org, en.wikibooks.org, beta.wikiversity.org, eo.wikipedia.org, nl.wikipedia.org, incubator.wikimedia.org
• Tuesday, October 4, 23:00-03:00 UTC — Stage 3: remaining wikis.

Wikis in Stage 1 and 2 may experience more issues, so we plan to focus our attention to those wikis during these periods, and be particularly responsive. If you’d like to help make sure we catch problems before we roll out to your wiki, please help us test, by trying out the test wiki starting Tuesday, and report the issues you find.

(more…)