Fortuna i forti aiuta, e i timidi rifiuta — an Italian proverb

The Wikimedia Foundation and Red Hat jointly organized the Second Open Source Language Summit on February 12th and 13th, 2013. The summit was held at the Red Hat engineering center in Pune, India. Similar to the previous summit, this face-to-face work session was focused on internationalization (i18n) and localization (l10n) features, font support, input method tools, language search, i18n testing methods and standards. The sessions were work sprints, each with special focus on a key area. Participants included core contributors from the Wikimedia Foundation, Red Hat (including Fedora SIG members), KDE, FUEL, Google and C-DAC. Below is a summary of what was accomplished during these two days.

During the summit, teams from different organizations came together to discuss language-related challenges, and worked together on features and tools to address them.

Input Methods

Parag Nemade and Santhosh Thottingal worked on making additional input methods available for the jQuery.IME library. 60 input methods, covering languages like Assamese, Esperanto, Russian, Greek, Hebrew were added bringing the total to 144. Also IMEs from the m17n library missing from the jQuery.IME library were identified.

Translation tools, translatewiki.net & FUEL Sprint

Siebrand Mazeland and Niklas Laxström, together with Ankit Patel, Rajesh Ranjan and Red Hat language maintainers, worked to identify more tools that could be used as Translation aids in a translation system. The FUEL project aims to standardize translations for frequently used terms, translation style and assessment methodology. Until now it has focused mostly on languages of India. The FUEL project can now be translated in translatewiki.net. Pau Giner demonstrated new designs for the translation editor and terminology usage, remotely from Spain.

Language Coverage Matrix

To better evaluate the needs for enabling support for languages, a matrix detailing the requirements and availability of basic and extended features is being drawn up. With 285 languages currently supported in Wikimedia and more than 100 in Fedora, this document will be instrumental in bridging the gaps and porting features across projects and platforms. Key areas of evaluation include input methods, fonts, translation aids like glossaries and spell-checkers, testing and validation methods, etc. A preliminary draft was created during the summit by Alolita Sharma, Runa Bhattacharjee and Amir E. Aharoni.

Fonts, WebFonts

An initiative to document the technical aspects of fonts for scripts for languages spoken in India started during the language summit. For each of the scripts, a reference font will be chosen and each font will be explained in detail to intersect with the Open Type font specification as a standard. It will aim to act as a reference document for any typographer working on Indian language fonts. Initial draft and outline of this document was prepared during the second day of the language summit, mainly by Santhosh Thottingal and Pravin Satpute.

Testing Internationalization Tools

Finding suitable methods for testing internationalized components and contents was the major focus of this sprint, with the Fedora Localization Testing Group (FLTG) and Wikimedia’s Language Engineering team sharing details of their testing methods. The FLTG conducts Test Days prior to Fedora beta releases with a test matrix targeted at specific core components, and Wikimedia uses unit tests for frequent testing of their development features. The FLTG showed its plans to integrate the screenshot comparison method for testing localized interfaces. This method will be useful for Wikimedia too. Extending the method for web-based applications and Wikimedia’s language requirements (e.g. right-to-left) were identified as areas for collaboration.

More news from the Language Summit can be found in the tweets, the session notes and the full report.

Runa Bhattacharjee, Outreach and QA coordinator, Language Engineering

Design changes to the Wikipedia mobile site include new navigation and updated typography.

This week you’ll see some changes to the look and feel of Wikipedia on your phone, as the mobile team moves features that were tested on our experimental Beta site onto our mobile gateway. The updated mobile site will include a navigation system that makes it easier to explore our content, as well as visual improvements aimed at increasing the readability of articles.¹

The new navigation system is designed to make mobile features and settings more discoverable, paving the way for the addition of new features. In the coming months, the mobile team will continue to experiment with and build contributory features. Whether it’s uploading a photo (as with the Wiki Loves Monuments Android app), watching changes to articles, or even editing, we want anyone to be able to pitch in and help make Wikipedia even better.

If you’re just browsing articles, the first thing you’ll notice is the change to layout and typography. We chose these new fonts for improved device compatibility, ease of scannability and reading, and, more generally, to better fit the high quality standards to which all Wikimedia projects strive.² Our designers will continue to focus on typography going forward, since text is the primary way readers and editors interact with the site.³

Improving how we serve content on the mobile web is crucial for reaching our goal of 4 billion pageviews per month by June 2013, and for providing Wikipedia to more readers in countries where mobile is the primary form of Internet access.

But we’re not done improving the mobile experience—please help us by providing feedback on the new look of the mobile site! You can also become a Wikipedia Beta tester by signing up to receive updates and test new experimental features before they go live.

Maryana Pinchuk, Associate Product Manager

1. The most recent set of updates will be available for users of our mobile website (such as the English Wikipedia on mobile).

2. Studies have shown that certain fonts can influence the perception of content in terms of quality and reliability.

3. The current set of changes are for Latin scripts. Readability improvements for non-Latin scripts will be carried out in cooperation with our localization team.

The Internationalisation/Localisation (i18n/l10n) team at the Wikimedia Foundation works on a set of tasks every two weeks. This post is about the team’s accomplishment over the past two weeks.

The Universal Language Selector(ULS) is designed to not only change the user interface language, but also to help the user set other language settings. ULS will contain input method and font settings along with it. The current version of ULS includes full integration with WebFonts and is now set up for testing at http://translatewiki.net. Let us know about bugs you find by reporting at Bugzilla. ULS now uses the Project Milkshake component jquery.webfonts. This means that when ULS is deployed, the WebFonts extension will be deprecated.

Language Settings in the ULS

Display settings dialog with option to choose font and to change the user interface language.

Other accomplishments

• Published a detailed plan for collecting metrics and impact measurement criteria for our work. Please provide us feedback after reviewing these criteria.
• Conducted a translation rally at translatewiki.net. According to preliminary results 180 translators contributed to 65,000+ translations on different Wikimedia related projects in 115 languages. 54 users will share the bounty sponsored by Wikimedia Nederlands, the Dutch Wikimedia chapter.
• Improved prototypes for Translate workflow user experience and completed user tests. You can check out test results. If you are a translator and wish to help us test the prototypes, please sign up.
• Met with the Wikimedia Deutschland Wikidata team about moving ahead with ULS for Wikidata language selection.
• The Georgian alphabet is now supported in Extension:Narayam.
• All Marathi wiki projects now have input method to Narayam deployed.
• Tim Starling wrote a php parser for the CLDR plural rule definitions. Some time back we had written a JavaScript parser for this. Soon the plural support in MediaWiki localisation messages will be based on CLDR plural definitions.

Please remember to join in at our sprint demo every 2 weeks to keep up with our latest work. The demos are every other Tuesday at 15:00 UTC. For those who could not attend, you can watch the latest demo  as well as all our old demonstrations are now available  in commons.

Srikanth Lakshmanan, Internationalisation/Localisation Outreach / QA Engineer

Every two weeks a development sprint is finished. Every two weeks we evaluate what we achieved, what went well and what went wrong. Many of the stories of sprint 10 can be found in Mingle (user:guest, password:guest). There you see the stories that were accepted or postponed.

The stories that ended happily are all over the place.

• The Ahirani language, a language of India that uses the Devanagari script in the same way as Marathi, is now supported for web fonts and input methods.
• When a translation administrator encourages or discourages the translation of a text, this will now be logged. This helps translators prioritize their activities.
• WebFonts now uses the MicroType Express font compression technology. This makes sending fonts to your browser go much faster.
• A translator can inform how he wants to be contacted and how often he can be contacted. In true agile fashion, the software that will make use of this will be written in a future sprint
• Some texts only need to be translated in selected languages because they will reach a specific public or because it will be used in software that supports a limited number of languages. New functionality enables a translation administrator to select these languages.
• We did a lot of code review; it gets done as it is part of our plan

A few stories did not end on a high note:

• Configuring one translation memory for all the wikis where the WMF needs translation took much longer. The idea was to build it first on Labs. This idea has now been shelved and it will be configured directly in production.
• A lot of work has gone in EasyTimeline. This was to make its functionality usable in other scripts and languages that are written from right to left. It works after a fashion and many issues have been resolved. Sadly the devil is in the details. Ploticus is a dependency for EasyTimeline and it has a bugs in creating  SVG output. There is no plan to fix this bug in Ploticus ourselves, but we are trying to find developers who can. Until then, we cannot have progress on this feature. Please let us know if you are interesting this issue for us.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant

When a text is written, when it is printed for a first time, it will have a contemporary look. When you then look at historic texts, at a first publication, you will notice the many details that show its age. It can be in differences in orthography, differences in vocabulary and also differences in the layout, the fonts used.

When sources are published in Wikisource, maintaining the atmosphere of the original text is very important. It is why the original orthography and vocabulary are maintained and with the availability of the  WebFonts extension there is a potential to use fonts that give this impression of age.

In the Office hours of the Localisation team, the question was raised if we could support cuneiform. The answer to that was that we can when there is a freely licensed font. We found a freely licensed cuneiform font and it is made available on the Wikis that support WebFonts. The bigger question however is about all the other scripts that are of  historic significance. This is of particular relevance to the Sanskrit Wikisource; the Sanskrit language is written in many scripts and it is only recent when the Devanagari script became the default script.

For sources like the Quran maintaining the original orthography and characters is an article of faith. It is for this reason that characters were added to Unicode because alternate representations of the same characters were missing. We do have a beautiful freely license font, the Amiri font and we would love to support it in MediaWiki but we are struggling with technical issues.

For the Wikimedia Localisation team, it is impossible to identify all the needs for fonts, for historic text representation. This is why we have language support teams. They know their language, they can identify a need and hopefully they can identify usable freely licensed fonts. When they do, we can and will support fonts. In the mean time we will continue our work on a unified language selector.  This will make the use of WebFonts easy and obvious. At this time it works, but it is hard work for you as a user.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant

When good people get together in a friendly, well organised setting like this weekend in Pune, many great things happen. Several MediaWiki developers had come to provide the many people new to MediaWiki with their expertise and guide people into its inner workings.

Many people worked on Wikimedia mobile and the SmartPhone software, others worked on MediaWiki and its extensions. Bugs got fixed and functionality got extended.

One of the surprises was two people working on the localisation for the Mongolian language. The inclusion of a web font that will support the Dzonka language is another.

Dzongkha is the official language of Bhutan and according to Ethnologue, the script used is either Tibetan script, Uchen style or the Tibetan script, Umed style. These scripts and styles are also used for the Tibetan language, it is not only Dzongkha that stands to benefit.

One of the highlights of the work on the SmartPhone app is support for scripts that are written from right to left, this is now “beta” functionality. The result of more people looking at the code was that several bugs received the attention needed to make them go away. Scrolling was one area that got attention; this results in a smoother user experience.

New input methods have been created for Punjabi transliteration and for an Gujarati input method to be included in Narayam. The continued collaboration with RedHat engineers ensures that our work benefits both MediaWiki and RedHat/Fedora. We do realise that there is still a lot to do and it is not only documentation. Additional work was done on the “visual on-screen keyboard” that was started at the previous hackathon in Pune, it still needs more testing and design work.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant

Consolidation was the name of the game for the past sprint for Wikimedia’s Localization team. A bug triage, testing, documentation and bug fixes were the activities designed to make our software more stable and more usable. When you read the bug triage report it becomes clear how much the devil is in the details; real native language expertise is needed to understand and assess the issues  we aim to solve. Read the report and you will see how much we rely on our community, on people like Srikanth and Nemo_bis.

Now that we are writing documentation in a central place, like here on the language statistics of the Translate extension, we are now able to provide you with a help text that is specific to the context. For the language statistics it is a help text about “statistics and reporting“. This functionality is ready but will become available in the deployment of January 30. You can help us and yourself by reading and understanding the text. Ask when you have questions and you can translate the text and make the text that much more your own.

Narayam is another extension that has been improved with user documentation. This documentation is completely new and it can effectively replace existing documentation. The existing documentation has the benefit of being written in the local language and we expect that what is written will be similar to the Narayam documentation. The language communities can then decide if they want to point to the local documentation. Like all our software, the Narayam documentation will be available for translation. Having the translation ready may be one of the considerations.

A lot of work is going into the description of the many input methods like the Inscipt layout for Assamese. These descriptions are “must have” help information when you do not know a particular keyboard layout by heart. They also provide a wonderful opportunity to verify if our implementation for a particular keyboard method is correct. This is yet another instance where native speakers can help us a lot.

Testing and coming to grips with the different tools was a major goal for this sprint. PHPunit and Qunit is what is used to test PHP and JavaScript and the tests developed are used in an environment called TestSwarm and Jenkins (respectively for PHP and JavaScript). As our team is so much into language support, we are learning what the limits are for testing for different languages and scripts.

All in all there may have been a slush and we have done a lot of code review, but we also managed to make sure that our functionality has gained stability for this and future releases. Additionally, work was done on grammar support for JavaScript, but the patch for that was stuffed in a bug report because of the slush, as the story was moved to the next sprint. Grammar support is what fills the gap in localization support between JavaScript and PHP and makes it available to any and all other developers.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant

Every two weeks a sprint and every week a deployment. The Localisation team aims to bring you new and updated functionality when we have it.

As you can see in the summary below, the focus this sprint has been very much on the Translate extension. Management of translations and the translation process is what we have worked on. When texts are translated in a Wiki, they often are only needed within a specific time frame; it is now possible to mark a text as no longer needing any effort. For many languages there are multiple people involved in the work flow for the creation of a document that is well written in translation. When they are to work well together, it helps when their work changes its state so that it is clear that for instance something has been proofread.

The person who manages the publication and distribution of a page needs work flow states to decide what more needs to be done and what is ready. To do this he can make use of states that already exist or define additional states. These states are available as local messages and are available for translation.

Translate extension features

• Message work flow states help translator translate, review and making ready for publication
• There is now a new message group for recent translations. This message group makes these states possible in translation
• Special:MyLanguage can now be used with language sub pages to be used as the default fall-back instead of providing an untranslated version
• Pages marked for translation can now be marked as “discouraged”. They will no longer show up in the usual places. This prevents translators from translating them needlessly.
• Added {{#translationdialog:title}} for creating a link to the translation dialogue

Translate bug fixes

• The flash of unstylized content effect is reduced
• Made the extension work without legacy JavaScript globals
• The summary row in Special:LanguageStats and Special:MessageGroupStats is no longer sorted with rest of the rows.
• Fixes to the sizing of the translation editor dialogue
• Fixed a fatal error that sometimes occured when translation page title used GRAMMAR and the page was viewed with English UI.

Miscelaneous changes

• Parserfunctions ifexist magic word Italian translation fixed to ‘ifexist’
• Narayam preference wording changes from disable to enable
• The WebFonts icon no longer overlaps with the menu text
• WebFonts preview allows you to preview a text with a font. You can download these freely licensed fonts to your system.
• GENDER and PLURAL support are now available for use in JavaScript.
• Consistence updates for grouppage-* messages, for LocalisationUpdate
• Fixing be-tarask grammar forms

Changes deployed last week

• WebFonts was deployed for the Bishnupria Manipuri language; it uses the Lohit Bengali font
• Support for gendered name spaces was deployed for the Russian wikis.

As always, you are welcome to have a look at our sprint backlog (user:guest password: guest) and bug us in bugzilla with whatever needs fixing.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant

Probably the most interesting highlight of today’s i18n deployment is the configuration of the Translate extension on MediaWiki.org. We have observed that on some wikis special pages exist that explain in the language of the Wiki functionality like Narayam or WebFonts. Such documentation is welcome on all MediaWiki installations where the functionality is used by people using the same language for their user interface.

For writing the documentation MediaWiki.org is the obvious platform. With the deployment of Translate we have the basis for writing and translating user documentation in a structured and organised way.

Narayam and WebFonts have been updated to the latest versions that have been tested on translatewiki.net. As Narayam and WebFonts are still very much a work in progress, we invite anyone to continue their testing at translatewiki.net . The changes are:

• menu appears only on click, not when hovering
• menu positions are now correct for RTL languages and do not go off screen any more
• Narayam and Webfonts support the Kannada script for the Tulu language on the Incubator

There are also some smaller fixes among them the change of the autonym for the Veps language to “Vepsän kel”.. The full details for all the changes is at revision 106667.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant

With a new sprint, new functionality for MediaWiki is identified to be deployed in two weeks time. There is room for dealing with issues to do with Narayam and WebFonts. Many of the new activities have to do with documentation, translation and feedback.

The sprint backlog in Mingle (user: guest password: guest)

What we hope for is that the feedback functionality that is now part of MediaWiki can be used to ask for feedback of MediaWiki features. It is obvious that the Wikimedia Localisation team cannot support all the 300+ languages that have their projects or exist in the incubator. What we can do is process the information we get from our language support teams. Figuring out how to do this is one of the goals for this sprint.

The use of Narayam and WebFonts will be helped a lot with documentation; “where to find that character on this keyboard mapping” or “what does an international keyboard look like” are questions looking for an answer. Determining how to document and what to translate is not all that obvious. With keyboard maps and fonts distributed as part of MediaWiki documenting on “the” wiki does not scale to other Wikimedia wikis and, MediaWiki wikis outside the Wikimedia Foundation are as much in need of documentation. When people start using MediaWiki because of such language support features we accomplish real support for a language.

For this sprint, these questions are looking for an answer and in the mean time the Translate extension will gain these new features:

• Documents that need translation can be grouped together; for instance all the Fundraiser messages or Wikimedia reports
• Documents can be marked as no longer needing translation
• Changes to the state of documents and translations will be logged and the log will be available for viewing
• Depending on the state of a document or a translation, attention can be drawn when there is a need for activity

User documentation needs translation and hopefully many of the algorithms used for the localisation of MediaWiki at translatewiki.net will equally apply for user documentation. Life will become a lot easier for all those people who administer MediaWiki and have only a basic understanding of English. We hope to deliver this in one of our future sprints.

Thanks,
Gerard Meijssen
Internationalization / Localization outreach consultant