Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Posts by Tim Starling

Search restored after leap second bug

At midnight UTC on July 1, Wikimedia’s search cluster stopped working. A “leap second” inserted by the NTP daemon at that time caused Java processes to lock up, including our Lucene search system. The same bug affected many other websites. Our engineers restored service in less than two hours.

Leap seconds are added to our clocks once every few years so that the sun will be directly overhead of the Royal Observatory in Greenwich at precisely 12:00. Some people believe that the desire to keep these two time standards synchronised is anachronistic, and that it would be better to let them drift apart for 600 years and then add a single “leap hour”. I’m sure many computer engineers would breathe a sigh of relief if such a change were implemented.

Tim Starling, Lead Platform Architect

MediaWiki 1.17.0

We are proud to announce the first stable release of the 1.17 series.

MediaWiki 1.17 is a very large release that contains many new features and bug fixes. This is a summary of the major changes of interest to users. You can consult the release notes for the full list of changes in this version.

What’s new?

PHP 5.2.3

We now require PHP version 5.2.3 or later. Why? Well, it brings with it some tools for your beloved developers. It was released on June 1, 2007, so we believe this requirement will not be a hassle for administrators. Be sure to check your PHP installation and contact your host if it runs an outdated PHP version.

New installer

The installer now supports many languages!

MediaWiki 1.17 is shipping with a completely redesigned installer to fix a lot of outstanding bugs, clean up the code quality, and make it easier to use. Notably, you can now run upgrades from the web without having to move LocalSettings.php. A couple of other notable changes:

  • The installer can now be fully localized like the rest of the software and contains numerous help dialogs.
  • The installer script directory has been renamed from config/ to mw-config/.
  • You now download your generated LocalSettings.php at install completion, rather than writing it straight to the configuration directory. The previous behavior was a security risk.
  • IBM DB2 and MSSQL support were dropped from the installer.

ResourceLoader

As web browsers have become more capable, the software that MediaWiki runs on them has become more complex. This trend has resulted in developers needing an efficient way to package and deliver code to web browsers.  To address this, MediaWiki 1.17 ships with ResourceLoader: a framework which combines and minifies CSS and JavaScript before delivering them to the web browser.  ResourceLoader improves performance, while also making it easier to write client-side features.  ResourceLoader allows developers to organize scripts, styles, and messages into named modules. Any number of modules can be loaded through a single request, improving page load times. Code is minified automatically and loaded when needed, reducing unnecessary downloads. Other advanced features include the ability embed images in style sheets using data URIs, or automatically flipping horizontal information in style sheets for right-to-left user interfaces.

Category sorting

Category sorting has been drastically improved.

  • Sorting is now case insensitive.
  • Sub-categories, pages and files can now be paged separately.
  • When several pages are given the same sort key, they sort by their names instead of randomly.

Language support

As with every release, MediaWiki 1.17 brings improved support for languages in MediaWiki, with improved translation and features for the many supported languages.

New languages:

  • Moroccan Spoken Arabic (ary)
  • Banjar (bjn)
  • Kabardian (Cyrillic) (kbd-cyrl)
  • Latgalian (ltg)
  • Minangkabau (min)
  • Dutch (informal) (nl-informal)
  • Rusyn (rue)

API

API bug fixes and new features have been added to 1.17, providing more options for input and output.

  • API output can now be formatted by PHP’s var_export() (format type is dbg/dbgfm).
  • An API module was added to list page properties.
  • PARAM_REQUIRED can now be used on parameters, to have the API enforce existence before code even reaches the module.
  • The API now has a Really Simple Discovery module, useful for publishing service information by the API.

API breaking changes

The API contains 3 breaking changes against previous releases:

  • action=patrol now requires POST.
  • The patrol token is no longer the same as edit token.
  • Session keys returned by ApiUpload are now strings instead of integers.

Other

  • Interwiki links in articles are now recorded in a separate table.
  • Users can now add CSS and JS to all skins by using User:<name>/common.css and User:<name>/common.js.
  • Oracle Database support has been improved, and is now ready for beta testing. If you work in an environment where Oracle is readily available, and you can’t get access to MySQL, this may be a useful alternative for you. Please try it out and let us know if it works for you. Oracle support is not yet recommended for use in production.

This blog post is based on the MediaWiki 1.17 wiki page on www.mediawiki.org, which was collaboratively edited. Please see the page history for credits.

MediaWiki version statistics

Some kind people at Qualys have surveyed versions of open source web apps present on the web, including MediaWiki. Here is the relevant page from their presentation:

MediaWiki versions 2010-07-30

For the original see:

And the press release:

They make the point that 95% of MediaWiki installations have a “serious vulnerability”, whereas only 4% of WordPress installations do. While WordPress’s web-based upgrade utility certainly has a positive impact on security, I feel I should point out that what WordPress counts as a serious vulnerability does not align with MediaWiki’s definition of the same term.

For instance, if a web-based user could execute arbitrary PHP code on the server, compromising all data and user accounts, we would count that as the most serious sort of vulnerability, and we would do an immediate release to fix it. We’re proud of the fact that we haven’t had any such vulnerability in a stable release since 1.5.3 (December 2005).

However in WordPress, they count this as a feature, and all administrators can do it. Similarly, WordPress avoids the difficult problem of sanitising HTML and CSS while preserving a rich feature set by simply allowing all authors to post raw HTML.

If you are running MediaWiki in a CMS-like mode, with whitelist edit and account creation restricted, then I think it’s fair to say that in terms of security, you’re better off with MediaWiki 1.14.1 or later than you are with the latest version of WordPress.

However, the statistics presented by Qualys show that an alarming number of people are running versions of MediaWiki older than 1.14.1, which was the most recent fix for an XSS vulnerability exploitable without special privileges. There is certainly room for us to do better.

We have a new installer project in development, which we hope to release in 1.17. It includes a feature which encourages users to sign up for our release announcements mailing list. But maybe we need to do more. Should we take a leaf from WordPress’s book, and nag administrators with a prominent notice when they are not using the latest version? Such a feature would require MediaWiki to “dial home”, which is controversial in our developer community.

Tim Starling, Lead Platform Architect

MediaWiki 1.16.0

We are proud to announce the first stable release of the 1.16 series. Selected changes that may be of interest since MediaWiki 1.15 are:

  • Watchlists now have RSS/Atom feeds. RSS feeds generally are now hidden, since Atom is a better protocol and is supported by virtually all clients.
  • It’s now possible to block users from sending email via Special:Emailuser.
  • The maintenance script system was overhauled. Most maintenance scripts now have a useful help page when you run them with –help.
  • AdminSettings.php is no longer required in order to run maintenance scripts. You can just set $wgDBadminuser and $wgDBadminpassword in your LocalSettings.php instead.
  • The preferences system was overhauled. Preferences are stored in a more compact format. Changes to site default preferences will automatically affect all users who have not chosen a different preference.
  • Support for SQLite was improved. Some broken features were fixed, and it now has an efficient full-text search.
  • The user groups ACL system was improved by allowing rights to be revoked, instead of just granted.
  • A new localisation caching system was introduced, which will make MediaWiki faster for almost everyone, especially when lots of extensions are enabled.

By default, this new system makes a lot of database queries. If your database is particularly slow, or if your system administrator limits your query count, or if you want to squeeze as much performance as possible out of Mediawiki, set $wgCacheDirectory to a writable path on the local filesystem. Make sure you have the DBA extension for PHP installed, this will improve performance further.

MediaWiki 1.15.5 was also released today. Both MediaWiki 1.15.5 and 1.16.0 contain important security fixes. For further details please read the release announcement.