Highlights from the latest development sprint of the Language Engineering team include the release of a new version of the MediaWiki Language Extension Bundle, and continued progress on Translation User Experience (UX) and the Language Coverage Matrix.

Screenshot of the redesigned proofread view for the Translate extension showing translations in Georgian.

Design and development improvements continued for Translate UX, also known as TUX. A preliminary implementation of the Proofreading feature (per the specifications in the design document) includes features to view the messages adjacently, adding clickable markers for proofreading and switching between proofreading and translation mode. Pau Giner presented these updates at an open session and also invited users to join the ongoing usability tests.

Amir Aharoni announced the release of MediaWiki Language Extension Bundle (MLEB) 2013.02. Besides localization updates in most of the components within MLEB, more features were added to Translate UX. The Universal Language Selector however had to be rolled back to the 2012.12 version to ensure compatibility with MediaWiki 1.20.

The Language coverage matrix document was updated to include more information about web fonts and input methods that are currently available for use in MediaWiki and Wikimedia projects. The document aims to provide an overview of the internationalization and localization support in languages across Wikimedia projects.

As part of the ongoing effort to use a CLDR-based, data-driven approach for internationalization features, plural rules for many languages were analyzed and custom rules were removed for a few languages.

The Language Engineering team will be hosting an IRC office hour session on Wednesday, March 13 2013 on in #wikimedia-office (FreeNode server) at 17:00 UTC. Topics will include discussion, questions, feedback about current projects, open bugs and projects planned for the next sprint.

Runa Bhattacharjee, Outreach and QA coordinator, Language Engineering

Starting Wednesday, March 13th, you’ll be able to make wiki pages even more useful, no matter what language you speak: we’re adding Lua as a templating language. This will make it easier for you to create and change infoboxes, tables, and other useful MediaWiki templates. We’ve already started to deploy Scribunto (the MediaWiki extension that enables this); it’s on several of the sites, including English Wikipedia, right now.

You’ll find this useful for performing more complex tasks for which templates are too complex or slow — common examples include numeric computations, string manipulation and parsing, and decision trees. Even if you don’t write templates, you’ll enjoy seeing pages load faster and with more interesting ways to present information.

Background

The text of English Wikipedia’s string length measurement template, simplified.

MediaWiki developers introduced templates and parser functions years ago to allow end-users of MediaWiki to replicate content easily and build tools using basic logic. Along the way, we found that we were turning wikitext into a limited programming language. Complex templates have caused performance issues and bottlenecks, and it’s difficult for users to write and understand templates. Therefore, the Lua scripting project aims to make it possible for MediaWiki end-users to use a proper scripting language that will be more powerful and efficient than ad-hoc, parser functions-based logic. The example of Lua’s use in World of Warcraft is promising; even novices with no programming experience have been able to make large changes to their graphical experiences by quickly learning some Lua.

Lua on your wiki

As of March 13th, you’ll be able to use Lua on your home wiki (if it’s not already enabled). Lua code can be embedded into wiki templates by employing the {{#invoke:}} parser function provided by the Scribunto MediaWiki extension. The Lua source code is stored in pages called modules (e.g., Module:Bananas). These individual modules are then invoked on template pages. The example: Template:Lua hello world uses the code {{#invoke:Bananas|hello}} to print the text “Hello, world!”. So, if you start seeing edits in the Module namespace, that’s what’s going on.

Getting started

The strlen template as converted to Lua.

Check out the basic “hello, world!” instructions, then look at Brad Jorsch’s short presentation for a basic example of how to convert a wikitext template into a Lua module. After that, try Tim Starling’s tutorial.

To help you preview and test a converted template, try Special:TemplateSandbox on your wiki. With it, you can preview a page using sandboxed versions of templates and modules, allowing for easy testing before you make the sandbox code live.

Where to start? If you use pywikipedia, try parsercountfunction.py by Bináris, which helps you find wikitext templates that currently parse slowly and thus would be worth converting to Lua. Try fulfilling open requests for conversion on English Wikipedia, possibly using Anomie’s Greasemonkey script to help you see the performance gains. On English Wikipedia, some of the templates have already been converted — feel free to reuse them on your wiki.

The Lua hub on mediawiki.org has more information; please add to it. And enjoy your faster, more flexible templates!

Sumana Harihareswara, Engineering Community Manager

Quem não arrisca não petisca — a Portuguese proverb

During their latest development sprint, the Wikimedia Language Engineering team conducted extensive review and testing of the Translate extension, and participated and contributed to two major open source events in India: a core developers Language Summit and GNUnify.

User experience improvements to the Translate tool will notably make it easier and more pleasant to translate content on Wikimedia sites that use it.

Translate Editor Updates

Progress continued on enhancements to the MediaWiki Translate extension. Further testing on the usability of the translation editor, search feature, and prototype of the advanced editing features were conducted by Pau Giner with five users from four different countries. The prototypes were tested in a great diversity of languages including Nepali, Chinese, Tetum, French, Breton, and Finnish. Based on this feedback, changes to the style and specifications for the prototype were made. Details about the individual tests can be found in the final report for this round of testing.

Community Participation

The Language Engineering team participated in the Open Source Language Summit and GNUnify, both held in Pune, India. The Open Source Language Summit, co-organized by the Wikimedia Foundation and Red Hat, consisted of work-sprints that focused on internationalization (i18n) and localization (l10n) features, font support, input method tools, language search, i18n testing methods and standards. More information about the event is available in the detailed event report.

The team also participated in GNUnify 2013, held at the Symbiosis Institute of Computer Studies and Research, in Pune. Besides presenting about the various projects that the team is currently working on, a translation sprint on translatewiki.net was also organized, as well as a workshop on jQuery.IME and a BoF session to discuss issues related to Wikimedia projects in Indian languages. Details of the accomplishments from the sessions at GNUnify 2013 can be found in the event report.

Other Achievements

Additionally, some changes to MediaWiki core were backported to support the newer version of the Universal Language Selector on MediaWiki versions 1.19 and 1.20. As there is no released maintenance version yet, MediaWiki Language Extension Bundle (MLEB) users are advised to remain on MLEB version 2012.12.

Focus for the next sprint

For the next development sprint, the team will work on more features for the Translate extension, like the proofreading mode and further improving the user experience. In addition to this, focus will be on putting together the language coverage matrix as a reference for the status of language support on MediaWiki, MediaWiki Extensions and Wikimedia projects.

Runa Bhattacharjee, Outreach and QA coordinator, Language Engineering

Lel the tacho pirrow, an’ it’s pars kaired — A Romani proverb

The Wikimedia Language Engineering team just completed its most recent development sprint, to introduce a new iteration of  the Translation Editor within the Translate Extension, and include features to make it more satisfactory as a translation workspace. The primary focus during this sprint has been to make the editor ready for production use. Some members of the team also attended FOSDEM 2013 in Brussels.

Translate interface features & enhancements

Paste Source Text  — Often found in translation editors, this feature allows for the source text to be easily copied over into the translation edit box. It’s now available in Translate and is particularly helpful when large portions of the source messages can be reused in the translation.

Message Documentation Display — Details about the messages within a project for translation can now be seen for all messages in a page, by picking the special “Message Documentation” language in the “Translate to” selector. This advanced option allows translators to view and evaluate the context for the messages that they are translating and also to see all the messages that were not documented yet.

The Message Documentation window of the MediaWiki Translate extension provides context for individual messages being translated.

Translation Editor UI  — The other enhancements that help translators to quickly review messages include:

1. Unchanged translations marked as “outdated” can be marked as suitable for use using the Confirm Translation button.
2. When translating a message, the translation aids of the subsequent message gets preloaded to avoid any delay during navigation.
3. Groups of messages, especially within Translation Pages with longer content, can now be set to a different state through a button click on a redesigned interface. This feature helps in identifying the Pages that can be pushed for publication.
4. Machine Translation suggestions from Apertium, Microsoft, and Yandex can now be dynamically presented for each message on the editor.

Besides the above, Search and Translation Editor were cross-integrated for translators, to edit Translations directly from the page displaying search results.

Search and Translation Editor were cross-integrated for translators, to edit Translations directly from the page displaying search results.

Pau Giner conducted a walkthrough of Translate user experience improvements, demonstrating the current state of development and the upcoming features for this extension.

Translate API changes — Changes to the Translate API now provide more information for the developers via the Web API, to help them implement customized translation interfaces.

During the development cycle, the team also engaged with the larger community to gather feedback about the new features through usability tests.

In other good news, jQuery.ime was successfully implemented on the Koha Library management system (v. 3.10) by Indranil Dasgupta. Also, do see this wonderful video about jQuery.ime by Chris Forno that blew us away.

Focus for the next sprint

Further enhancements to Translate continue to be the main focus for the next sprint. This includes review and testing of the latest designs. The Language Engineering team hopes to have more interaction in this regard at the Open Source Language Summit (organised and hosted in collaboration with Red Hat) and at GNUnify in Pune, India.

Runa Bhattacharjee, Outreach and QA coordinator, Language engineering

Batti il ferro finché è caldo —an Italian proverb

In its last two-week sprint, the Wikimedia Language Engineering team worked with developers from other teams to improve its keyboard support and we continued working on the new user interface for the Translate extension.

Input methods: More languages and support for mobile devices added

jquery.ime, Wikimedia’s portable keyboard layouts library got boosts from two sources during the last sprint.

Yuvi Panda from the mobile team refactored Wikimedia’s keyboard layouts library, jquery.ime, to make it usable on mobile phones. Now, over 60 keyboard layouts that are supported by IME will also be usable on Android mobile phones. If you’d like to try an early testing version of the mobile keyboard layouts and help developing them, head to the mobile keyboard layouts GitHub repository.

Engineers from Red Hat also joined the input method development effort and added new and improved layouts for the Gujarati, Punjabi, Tamil, Malayalam, Kannada and Telugu languages, spoken by millions of people in India.

If a keyboard layout for your language is missing, you can send a pull request to the main jquery.ime repository.

Progress on translation editor user experience

The team continued fixing and improving the new translation editor, getting it ready to release. Some of the recent improvements include:

• The most relevant translation memory suggestions are shown at the top.
• Messages are loaded automatically when the user scrolls to the bottom of the page.
• The status bar at the bottom of the page shows information about the status of the translations.
• Recently translated projects are now displayed correctly in the project selector.
• Discouraged translation projects are omitted from the group selector.
• Message documentation can now be edited inside in the translation editor.

The features that were already implemented, were also tested with real users by the team’s interaction designer Pau Giner. The issues that the users reported were noted and will be fixed in the coming sprints.

Niklas Laxström implemented Faceted search for translations using the Free Apache Solr engine and deployed an experimental version of the translation search on the testing site. He also made an open presentation about Solr and its upcoming use in translatewiki.net. You can watch Niklas’ presentation about Solr on YouTube.

Next week some of the team members are going to participate in the FOSDEM conference, and after that—the 2nd Language Summit in the Red Hat offices in Pune.

Amir Aharoni. Software Engineer (Internationalization), Language Engineering team

De mica en mica s’omple la pica i de gota en gota s’omple la bota. —a Catalan proverb

During its most recent development sprint, the Wikimedia Foundation’s Language Engineering team continued to improve the user experience of the Translate extension to make it as smooth and efficient as possible. Highlights include:

• Pressing the “Save” button immediately shows the next string to translate, while the saving is performed in the background.
• When progressing to the next translation, the page smoothly scrolls up.
• Explanations about translatable strings are shown beside the corresponding message in a convenient box, which becomes expandable if the documentation is too long.
• Machine Translation was made available for suggested translations.
• The differences between older versions of translatable strings are also shown in a new expandable box.
• The Language Selector API was updated to allow displaying all the documentation strings.
• The Solr search engine schema was tweaked to make searching translatable strings more efficient and feature-rich by offering faceted search.

Below is a brief demo of the latest features of the translation editor in action. You can see translating the Etherpad Lite project into Russian there.

The Language team also continues to work on squashing bugs and adding prioritized features. You can check out the latest bleeding edge version of the translation editor on translatewiki.net, or go back to the stable translation editor. Please report Translate bugs in Bugzilla.

Amir E. Aharoni, Software Engineer (Internationalization)

(Emor me’at ve-ase harbe. —a Hebrew proverb)

The Wikimedia Foundation’s Language Engineering team is continuing the makeover of the Translate extension, which started taking shape in early December. (Introduced in 2011, this MediaWiki feature powers the translation of Wikipedia’s software, announcements, reports and fundraising banners, and of other sites and software projects.)

During its latest two-week sprint, the team improved the actual interface used for submitting and editing translations:

A screenshot of the new work-in-progress translation editor

Information about the message and translations to other languages are now shown in a collapsible box on the right side of the translation area. Warnings about potential errors in the message are shown in a small box above the editing area, which is expandable, too.

The functionality for saving and skipping messages was updated. Usability testing observations by Arun Ganesh and Pau Giner suggest that users facing a hard part in a translation are more likely to just skip it than to report the problem. Because of this, skipping a message is now recorded and frequently skipped messages will be considered for re-wording.

In the next sprint the team will work on polishing the translation interface further: better display of documentation, translation suggestions and diffs, better responsiveness, more robust language selection and other features.

In other Language Engineering news:

• The December 2012 version of the MediaWiki Language Extension Bundle was released.
• Better support for language variants and alternative language codes was added to the Universal Language Selector.

Amir E. Aharoni, Software Engineer (Internationalization)

(This guest post comes from David Narváez, one of the organizers of Wiki Loves Monuments in Panamá. You can see the winning photos from the country here.)

El Cementerio de Corozal, 2nd Place, WLM Panamá

I started off by building our site using WordPress, the blog and content management system every volunteer in the team learned to tweak and love. We could have gone with other free software content management systems we are more used to, like Drupal, but we figured out that using the same technology most other countries were using for their WLM sites would help us leverage theme expertise from larger organizations.

WordPress provided a large set of plugins that helped us build a very dynamic website with just a few clicks, and most important of all, we were able to provide it in Spanish almost without any effort thanks to the great work of the internalization team for WordPress.

With the site online, we started thinking about putting the large plugin database to good use. One thing that came up was the idea to show the list of monuments that we built for WLM Panama in an interactive map using Open Street Map. While the toolbox for Wiki Loves Monuments already included such a map, by the time we started looking for a way to display our monument list, the WLM map did not work with our list. Because we needed to put the map online sooner rather than later, we decided not to wait for the toolbox map. We found the Fotomobil.at OSM Plugin, which lets you provide a text file that specifies all marks in the map, but we still needed an automated way to build the list of markers from the Mediawiki table on Spanish Wikipedia. After a lot of search (and I mean a lot of search), we found the Pijnu/Mediawiki-Parser tandem to parse the Mediawiki text and look for title, image, location and description of the monuments.

Once the site was complete, the next challenge was to develop a jury tool that would let juries browse the photos in the two categories we had, with monuments grouped by geographical location, EXIF metadata information for each photo and a simple account system. I ended up writing a web application, heavily based on the Dojo Javascript Toolkit. The database with the information for this tool was also provided by parsing the information from our list of monuments, but with the additional requirement of actually fetching the photos in various sizes. For this task, I went back to a project I was already familiar with: the MediaWiki API Client for Python.

One last task we needed to undertake was to move all of the photos uploaded to the contest through Flickr to Commons. Our obvious way to do this was to use the Flickrripper bot from the Python Wikipedia Robot Framework. Despite the issues we found with Flickr’s API, we managed to do a lot of our task using a patched version of Flickrripper. And speaking of API failures, I must add that during the whole process we never had a single glitch, or an unmet requirement from Wikimedia’s API, so I must stand up and congratulate the team of system administrators behind the Wikimedia projects.

In the aftermath of the event, after enjoying a very interesting prize ceremony, reading articles about WLM in our local newspapers, taking a break from the hard work around WLM and planning for next year, I’m starting to take on the large path of patches, integration code and development to collaborate with the many Open Source projects that made this a successful event for us. We are also contributing with donations to some of the projects that were key to our success, and expect to keep this tradition in years to come.

David E. Narváez, Wiki Loves Monuments, Panamá

Ei kannata mennä merta edemmäs kalaan. —a Finnish proverb

The Translate extension, a central piece in the puzzle that makes Wikipedia and the community around it massively multilingual, is getting a major overhaul.

“Translate”, as it’s commonly called, powers the translation of Wikipedia’s software, announcements, reports and fundraising banners, and of other sites and software projects. It focuses on making the translators’ work easy, efficient and, if possible, fun. The software gets frequent under-the-hood updates, and now the time has come for a major overhaul of its most visible part: the translation user interface.

Arun Ganesh and Pau Giner, from the Wikimedia Foundation’s Language Engineering team, have studied the current translation workflow by testing the software and interviewing translators. They drafted new interface ideas and tested experimental designs with users who speak different languages and have different levels of experience with the translation functionality.

In the team’s thirtieth two-week coding sprint, which ended last Tuesday, two major components of the overhaul have taken shape: the message group selector and the list of translatable messages.

The Message group selector. Message groups are groups of related translatable messages: a software project, a multilingual blog post or announcement, etc.

The group selector helps a translator find a project to translate using a tree-like structure of groups and sub-groups. Every project shows the completeness of the translation using a colorful progress bar. For quick and easy access to the projects that interest the translator, there’s a tab that shows recently used projects, and a responsive search function.

Listing of translatable interface message for the Visual Editor. Some messages are translated to French and some need review.

The list of strings to translate has been redesigned to improve clarity, making it easier to scan and distinguish between messages that are translated, untranslated and need to be updated (“fuzzy”).

The development of the improved user experience continues. In the next sprints, the team will complete these features and add new ones, such as an improved sign-up process and better search. Usability testing efforts will continue to ensure that the new designs provide an improved experience. If you are interested in trying the new translation tools, please volunteer for our usability testing sessions.

Other ways to connect with the Language Engineering team:

• Pau Giner and I will present on multilingual user testing and internationalization “dos and don’ts” in the live broadcast Wikimedia Open Tech Chat on Thursday, December 13 at 20:30 UTC.
• We’ll hold IRC Office Hours on Wednesday, December 17 at 17:30 UTC. Topics of discussion will be the translation user experience improvements, Universal Language Selector and general Q&A.

Amir E. Aharoni, Software Engineer (Internationalization)

“Live bird”, an illustration by Kim ‘Isarra’ Schoonover, one of Wikimedia’s interns for the Outreach Program for Women.

I’m glad to announce that Kim Schoonover, Mariya Miteva, Priyanka Nag, Sucheta Ghoshal, Teresa Cho and Valerie Juarez will join the MediaWiki community as full-time interns between January and March 2013. They have been selected as part of the FLOSS Outreach Program for Women, an initiative of the GNOME Foundation, with participation by several other free software projects: Deltacloud, Fedora, JBoss, Mozilla, Open Technology Institute, OpenITP, OpenStack, Subversion, Tor and Wikimedia. A total of 25 women have been selected in this edition.

Each MediaWiki intern will work on a specific project:

• Kim Schoonover (aka ‘Isarra’) is a Wikipedia contributor based in Colorado (USA) with an interest in user experience and design. She has contributed a few patches to MediaWiki projects already. She will work on design documents for Flow, the planned UI for user discussion pages, mentored by Brandon Harris and Heather Walls.
• Mariya Miteva is a Masters of Information Systems Student in New York (USA), although she is originally from Sofia (Bulgaria) and will conduct her internship from there. She will find MediaWiki vendors and define the best ways to work with them, and will also have a backlog of product management tasks. Her mentors are Sumana Harihareswara and James Forrester.
• Priyanka Nag is a Computer Science post-graduate student from Pune (Maharashtra, India). She will work in the documentation of noteworthy local templates, gadgets and styles used in various Wikimedia projects, mentored by Amir E. Aharoni.
• Sucheta Ghoshal is a Computer Science and Engineering student based in Kolkata (West Bengal, India), an active Wikipedia contributor and member of the Wikimedia Kolkata meetup. She will work on the productization of EtherEditor MediaWiki extension. She will be mentored by Mark Holmquist.
• Teresa Cho has double major in Computer Science and Economics and she lives in Redwood City (California, USA). Her project consists of developing a MediaWiki extension to interact with Git repositories, and is mentored by Sébastien Santoro.
• Valerie Juarez has a BS in Computer Science and she lives in Sour Lake (Texas, USA). She will help managing the bug reports, organize a couple of “Bug Days” and will develop a proposal for streamlining user feedback from multiple Wikimedia sources. She will be mentored by Andre Klapper and Quim Gil.

Our mentors assisted about 25 women interested in the MediaWiki community during the application process. From those, 10 submitted full applications, including a project proposal and a microtask completed as part of the submission. We were impressed by the quality of many submissions, but we couldn’t take more due to lack of related mentors and funding. We encourage all applicants to stay around in the MediaWiki community, since we have no doubt they can all become top contributors and have future opportunities.

In addition to the availability of specialized mentors and the quality of the proposals, our selection criteria took into account the diversity of profiles, origins and previous involvement in Wikimedia projects. We had a rather open process for submissions and selection of candidates that gave participants a chance to check other proposals, improve their own and receive public endorsements.

We wish a happy landing to our new interns and the best of luck in their projects! You’ll be hearing more from them over the next few months.

Quim Gil, Technical Contributor Coordinator