Major news in January include:

• the successful migration of our main services to our data center in Ashburn, Virginia;
• new features available in our mobile beta;
• progress on input methods and our upcoming translation interface;
• the announcement of GeoData, a feature to attach geo-coordinates to Wikipedia and Wikivoyage articles;
• a testing event to assess how VisualEditor handles non-Latin characters.

Note: We’re also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.
(more…)

(Update on January 22nd, 2013, 20:00 (UTC): Our Operations team considers the migration to be over. Major disruption is no longer expected.)

All Wikimedia sites, including Wikipedia, may encounter temporary interruptions on January 22–24, as they transition to servers in a new data center in Ashburn, Virginia (see more photos).

Next week, the Wikimedia Foundation will transition its main technical operations to a new data center in Ashburn, Virginia, USA. This is intended to improve the technical performance and reliability of all Wikimedia sites, including Wikipedia.

Engineering teams have been preparing for the migration to minimize inconvenience to our users, but major service disruption is still expected during the transition. Our sites will be in read-only mode for some time, and may be intermittently inaccessible. Users are advised to be patient during those interruptions, and share information in case of continued outage or loss of functionality.

The current target windows for the migration are January 22nd, 23rd and 24th, 2013, from 17:00 to 01:00 UTC (see other timezones on timeanddate.com).

Wikimedia sites have been hosted in our main data center in Tampa, Florida, since 2004; before that, the couple of servers powering Wikipedia were in San Diego, California. Ashburn is the third and newest primary data center to host Wikimedia sites.

A major reason for choosing Tampa, Florida as the location of the primary data center in 2004 was its proximity to founder Jimmy Wales’ home, at a time when he was much more involved in the technical operations of the site. In 2009, the Wikimedia Foundation’s Technical Operations team started to look for other locations with better network connectivity and more clement weather. Located in the Washington, D.C. metropolitan area, Ashburn offers faster and more reliable connectivity than Tampa, and usually fewer hurricanes.

The Operations team started to plan and prepare for the Virginia data center in Summer 2010. The actual build-out and racking of servers at the colocation facility started in February 2011, and was followed by a long period of hardware, system and software configuration. Traffic started to be served to users from the Ashburn data center in November 2011, in the form of CSS and JavaScript assets (served from “bits.wikimedia.org“).

We reached a major milestone in February 2012, when caching servers were set up to handle read-only requests for Wikipedia and Wikimedia content, which represent most of the traffic to Wikipedia and its sister sites. In April 2012, the Ashburn data center also started to serve media files (from “upload.wikimedia.org“).

Cacheable requests represent about 90 percent of our traffic, leaving 10 percent that requires interaction with our web (Apache) and database (MySQL) servers, which are still being hosted in Tampa. Until now, every edit made to a Wikipedia page has been handled by the servers in Tampa. This dependency on our Tampa data center was responsible for the site outage in August 2012, when a fiber cut severed the connection between our two locations.

Starting next week, the new servers in Ashburn will take on that role as well, and all our sites will be able to function fully without relying on the servers in Florida. The legacy data center in Tampa will continue to be maintained, and will serve as a secondary “hot failover” data center: servers will be in standby mode to take over, should the primary site experience an outage. Server configuration and data will be synchronized between the two locations to ensure a transition as smooth as possible in case of technical difficulties in Ashburn.

Besides just installing newer hardware, setting up the data center in Ashburn has also been an opportunity for architecture overhauls, like incremental improvements of the text storage system, and the move to an entirely new media storage system to keep up with the growth of the content generated and curated by our contributors.

Wikimedia’s technical infrastructure aims to be as open and collaborative as the sites it powers. Most of the configuration of our servers is publicly accessible, and the Wikimedia Labs initiative allows contributors to test and submit improvements to the sites’ configuration files.

The Wikimedia Foundation currently operates a total of about 885 servers, and serves about 20 billion page views a month, on a non-profit budget that relies almost entirely on donations from readers.

Guillaume Paumier
Technical Communications Manager

Major news in December include:

• The launch of an alpha, opt-in version of the VisualEditor to the English Wikipedia, a project more complex than it appears;
• A research study on the use of the Article Feedback feature;
• New metrics for the MediaWiki community;
• The start of the Outreach Program for Women;
• Continued work to improve the workflow and interface for translators.

Note: We’re also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

(more…)

Major news in November include:

• the soft-launch of Wikivoyage.org;
• the launch of a new HTML5 video player, with support for WebM video, subtitles and multiple resolution derivatives;
• the first version of the Analytics Hadoop cluster put into service;
• the opening up of the product management process to volunteers.

Note: Like last month, we’re proposing a shorter and simpler version of this report for less technically savvy readers.

(more…)

A new video player has been enabled on Wikipedia and its sister sites, and it comes with the promise of bringing free educational videos to more people, on more devices, in more languages.

The player is the same HTML5 player used in the Kaltura open-source video platform. It has been integrated with MediaWiki (the software that runs Wikimedia sites like Wikipedia) through an extension called TimedMediaHandler. It replaces an older Ogg-only player that has been in use since 2007.

The new player supports closed captions in multiple languages.

Based on HTML5, the new player plays audio and video files on wiki pages. It brings many new features, like advanced support for closed captions and other timed text. By allowing contributors to transcribe videos, the new player is a significant step towards accessibility for hearing-impaired Wikipedia readers. Captions can easily be translated into many languages, thus expanding their potential audience.

TimedMediaHandler also comes with other useful features, like support for the royalty-free WebM video format. Support for WebM makes it possible to seamlessly import videos encoded to that format, such as freely-licensed content from YouTube’s massive library.

Even further behind the scenes, TimedMediaHandler adds support for server-side transcoding, i.e. the ability to convert from one video format to another, in order to deliver the appropriate video stream to the user depending on their bandwidth and the size of the player. For example, support for mobile formats is available, although it is not currently enabled.

Sponsored by Kaltura and Google, developers Michael Dale and Jan Gerber are the main architects of the successful launch of the new player. With the support of the Wikimedia Foundation’s engineering team and Kaltura, they have gone through numerous cycles of development, review and testing to finally release the fruits of years of work.

Efforts to better integrate video content to Wikipedia and its sister sites date back to early 2008, when Kaltura and the Wikimedia Foundation announced their first collaborative video experiment. Since then, incremental improvements have been released, but the deployment of TimedMediaHandler is the most significant achievement to date. (more…)

Major news in October include:

• a redesign of the mobile site emphasizing readability and navigation;
• the launch of a Wikipedia app for Windows RT and Windows 8 tablets;
• the test of a redesigned account creation page.

Note: As of last month, we’re proposing a shorter and simpler version of this report for less technically savvy readers.

(more…)

Major news in September include:

• The introduction of translation memory by the Language engineering team;
• A breakage of our gerrit tool, and its subsequent recovery;
• A new version of the Wiki Loves Monuments App, that allows users to use it in combination with a separate camera.
• A new EPUB export feature enabled on the English Wikipedia;
• A call to nonprofits, schools, libraries, etc. who may be interested in getting decommissioned Wikimedia servers;
• The launch of the Page curation feature on the English Wikipedia;
• The completion of Google Summer of Code 2012. (more…)

Major news in August include:

• A site outage caused by a fiber cut between our two data centers;
• Progress by the Internationalization team on the Universal language selector & Milkshake, and WebFonts;
• Changes in our analytics system to improve the accuracy of the active editors metric;
• Major work on the Wiki Loves Monuments app.

Note: Following a reader’s advice, we’re trying out slightly colored backgrounds to help readers skim through sections. Let us know in the comments how that works for you, and how to improve it for the next report. (more…)

Major news in July include:

• Engineering presence at Wikimania 2012 in Washington, D.C., and the pre-Wikimania hackathon;
• the launch of Limn, an open source dataviz toolkit developed by the Wikimedia analytics team;
• the deployment of Article Feedback Version 5 (which supports free-text feedback and moderation thereof) to 10% of English Wikipedia articles

(more…)