Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Posts by Guillaume Paumier

Wikimedia engineering May 2012 report

Major news in May include:

(more…)

Diverse Wikimedia tech crowd gathers in Berlin

A diverse crowd of engineers, volunteer developers, template writers, gadget maintainers and bot operators have gathered in Berlin for this coding event.

The Wikimedia technical community is gathering this week-end in Berlin to code, discuss, learn and generally improve the infrastructure and tools behind Wikipedia and its sister sites.

The “Berlin Hackathon 2012“, a yearly coding event happening in the German capital since 2009, is co-organized by Wikimedia Deutschland. It is taking place this year at STATION Berlin, a former postal train station now converted to an exhibition and event hall. It was preceded by a two-day summit on Wikidata and RENDER, projects aiming to integrate structured data with Wikipedia.

The crowd of about 130, coming from 30 countries, includes Wikimedia Operations engineers, who maintain the site’s hardware and network infrastructure, and MediaWiki developers, who build and improve the software powering all Wikimedia sites. Topics of discussion and workshops include the new code review workflow after the move of the primary code repository to Git, in an attempt to facilitate contributions.

“This is the largest Wikimedia coding event we’ve ever had,” started Sumana Harihareswara, Engineering Community Manager at the Wikimedia Foundation, as she opened the event. Sumana, the lead organizer of the event, has reached out to many Wikimedian technologists outside the core circle of MediaWiki developers, to make the event more inclusive. Indeed, many attendees are joining a Wikimedia coding event for the first time.

The hackathon focuses on recent developments that are enabling more community-driven innovation.

For example, expert Wikipedia editors have been invited to learn about Lua scripting, which is poised to replace or supplement MediaWiki templates later this year.

Templates, originally introduced as a way to embed standardized texts in several Wikipedia articles, have been combined by Wikipedians with parser functions to create a limited programming language. They have become a performance bottleneck for long Wikipedia articles embedding many of them, like Barack Obama, for which a new version of the page can take up to 40 seconds to be generated by the server.

By replacing complex templates by simpler ones augmented with Lua scripts, developers aim to provide editors with a proper scripting language that will be both more powerful and more efficient than ad-hoc ParserFunctions-based logic. Tutorials are being held in Berlin to help end-users learn about Lua and how to adapt templates to this new technology.

Another group of attendees is “Gadgets” maintainers, who have come to Berlin to learn how to adapt their tools to a new version of the software, called “ResourceLoader 2.0“. It will make it possible to centralize custom JavaScript snippets developed by editors, and share them across Wikimedia sites for other communities to use.

Developers and engineers, used to collaborate online, are also using this opportunity to socialize and discuss face-to-face.

“We’ve never had so much activity in our technical community” explains Erik Möller, VP of Engineering and Product Development. “In Berlin, we’re systematically raising awareness of all the recent developments that are enabling more community-driven innovation than ever before. It’s a great time to be a Wikimedian.”

Besides coding, workshops and presentations, the event is also an opportunity for Wikimedia technologists, who usually collaborate over the internet, to mingle and socialize. They will next meet in July in Washington, D.C., for the annual Wikimania conference and its very own Hackathon. By then, they are expected to unveil the first working prototype of the Visual Editor, the upcoming user-friendly interface to edit Wikipedia articles.

Guillaume Paumier
Technical communications manager

Wikidata Summit kicks off in Berlin

The 2-day event is focusing on Wikidata and RENDER, technologies to integrate structured data with Wikipedia and its sister sites.

The Wikidata & RENDER summit, a 2-day technical event focusing on the integration of structured data with Wikipedia, started today in Berlin, Germany, as a prologue to the Wikimedia Hackathon.

The event, organized by Wikimedia Deutschland, consists of workshops, presentations and coding, split into two tracks: one on Wikidata, and the second on RENDER.

The Wikidata project was announced earlier this year; its goal is to build the software infrastructure to support a common source of structured data that can be used in all Wikipedia articles, regardless of their language.

It would work in the same way that images and other multimedia content from Wikimedia Commons can be embedded into any page on a Wikimedia site.

Wikidata is expected to lead to a higher consistency and quality within Wikipedia articles, increased availability of information in the smaller language editions, and decreased maintenance effort for Wikipedia volunteers.

RENDER, the other focus of this summit, is a EU-funded project aimed at developing methods, techniques, software and data sets for scholars and readers (such as Wikipedia users) to understand, describe, process and make use of the diversity of knowledge and information.

About fifty people were invited to attend: they are Wikimedia Deutschland engineers, Wikimedia Foundation engineers, and volunteer MediaWiki developers, with expertise in structured data, MediaWiki and Wikimedia projects.

About 50 engineers and volunteer developers have gathered in Berlin for this prelude to the Wikimedia hackathon.

Sessions will be held today and tomorrow at Station-berlin – Hall 6, the same venue where the Berlin Hackathon 2012 (a.k.a. “Wikimedia Dev days”) will take place, starting tomorrow evening.

Follow and participate

We don’t have live video streaming of the event, but you can follow what’s happening on site through a variety of channels:

  • participants are taking live collaborative notes that will be posted on wiki when sessions are over;
  • they’re also posting information snippets on Twitter and Identi.ca; join the discussion with the #wikidata and #RENDER hashtags;
  • last, you can join us on IRC in the #wikimedia-wikidata and #mediawiki channels on Freenode.

Let us know on IRC or in the comments below if we can do anything else to let you participate remotely.

Guillaume Paumier
Technical communications manager

New book dives into the architecture of MediaWiki, git, puppet and other open-source applications

The cover of the book, based on the photo of a building from a low-angle shot

The Architecture of Open-Source Applications is a collection of technical essays detailing the architecture of twenty-four major open-source applications.

The second volume of the Architecture of Open-Source Applications book, which includes a chapter on MediaWiki, is now available online and on lulu.com.

The Architecture of Open-Source Applications is a collection of technical essays detailing the architecture of twenty-four major open-source applications. This is the second volume of a series that aims to help developers understand how great and large programs are constructed, and the decisions (or accidents) that led to the way they now work. The series draws inspiration from books used by architects that feature case studies of the great buildings of history.

This volume contains a chapter detailing the inner workings of MediaWiki, the wiki software that powers all Wikimedia sites, including Wikipedia.

The writing of the chapter was coordinated by myself and Sumana Harihareswara. While I put together the majority of the content, it wouldn’t have been possible without the initial knowledge-sharing effort made by many Wikimedia engineers and volunteer MediaWiki developers, who also reviewed and improved the several revisions the text underwent.

The chapter on MediaWiki is available on the book’s website, along with the other chapters from both volumes. Its content was integrated into the documentation on mediawiki.org (at MediaWiki history and Manual:MediaWiki architecture) when it was completed in November 2011.

Greg Wilson and Amy Brown, the book’s editors, contacted the Wikimedia Foundation in August 2011 to offer to feature MediaWiki in the second volume. We chose a very collaborative approach to writing the chapter to ensure that the content was accurate and thorough, and also to split the workload among subject matter experts.

This volume dives into the inner workings of other tools familiar to the Wikimedia community, like Git, GNU Mailman, nginx and Puppet.

All of the book’s content is released under the Creative Commons Attribution license, similar to the license used on Wikimedia sites. It is freely available for reading online at http://www.aosabook.org, and you can also order a print from lulu.com. E-book and PDF versions will be available for purchase shortly. All royalties from purchases are donated to Amnesty International.

This is the second book published this year that contains a chapter written by Wikimedia staff, after the publication of Open Advice, a collection of essays, stories and lessons learned by members of the Free Software community.

I hope the chapter on MediaWiki, and also the rest of the book, will prove useful and interesting to the Wikimedia community and other developers. If you enjoyed it, learned from it, or would like to see more publications of this type, let us know!

Guillaume Paumier
Technical communications manager

Wikimedia engineering April 2012 report

Major news in April include:

Wikimedia engineering March 2012 report

Major news in March include:

(more…)

Wikimedia engineering February 2012 report

Major news in February includes:

  • The difficult deployment of our Swift infrastructure to serve image thumbnails
  • Continued success for our Wikipedia Android app
  • The deployment of MediaWiki 1.19 to all Wikimedia sites except for most Wikipedia languages
  • Continued preparation for our move from Subversion to git

(more…)

MediaWiki 1.19 deployment to Wikimedia sites: Test it before it breaks

The logo of MediaWiki (a yellow sunflower surrounded by two pairs of blue square brackets) with gradients symbolizing its coming to age for the next version

Wikimedia sites will gradually be upgraded to version 1.19 of MediaWiki over the second half of February 2012.

This article is available in other languages on mediawiki.org.


Wikimedia engineers are putting the final touches to the latest version of MediaWiki, the software that powers Wikipedia and its sister sites. This version, labeled “1.19wmf1″, will be deployed to Wikimedia sites in stages, starting next week.

We’ve recently set up a Beta cluster, replicating a selection of Wikimedia wikis, where Wikimedians have tested the new version and checked that it worked reasonably well with their local wiki’s specific customizations.

Things are looking good, and the current plan is to run the deployment in five stages between February 15th and March 1st, 2012. The schedule may change based on unexpected issues, so you should refer to the MediaWiki 1.19 roadmap for an up-to-date schedule of when your wiki will be affected. (more…)

Wikimedia engineering January 2012 report

Major news in January include:

(more…)

Free software community shares lessons learned in “Open Advice” book

Open Advice book cover

The "Open Advice" book is available for free download, or purchase as print from lulu.com.

The Open Advice book, a collection of essays, stories and lessons learned by members of the Free Software community, is out!

The book was just announced at FOSDEM, the Free and Open Source Software Developers’ European Meeting, in Brussels over the week-end.

About 50 authors from many different projects of the free software community were brought together by Lydia Pintscher, the book’s editor, who started the project in early 2011.

A year and 380 pages later, the book is now available, and tries to provide an answer to the question: What’s the key thing you would have liked to know when you started contributing?

Authors answer that question for many topics, ranging from “Writing patches” to “Documentation for Novices”, to business models, conferences, translation, design, and more.

I contributed “Learn from your users”, a chapter on user experience and usability testing. You’ll also recognize other names from the Wikimedia community, like Evan Prodromou, Markus Krötzsch and Felipe Ortega.

You can learn more about the book and the authors on the book’s website.

All the content of the book is released under the same license as Wikipedia, the Creative Commons Attribution Share-Alike license.

Check it out! You can download the book for free as a PDF file, order a print from lulu.com if you prefer paper books, or fork the text on GitHub.

I hope you’ll like the book, and it’ll prove useful, whether you’re new to the world of software, or you’re a seasoned contributor already.

Guillaume Paumier
Technical Communications Manager