Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Posts by James Forrester

The alpha version of the VisualEditor is now in 15 languages

This post is available in 6 languages: English 100% Deutsch German • Français 7%Español 7%Svenska7%日本語 7%На русском языке 7%

English

Today the Wikimedia Foundation launched an alpha, opt-in version of the VisualEditor to fourteen Wikipedias, which follows our release to the English Wikipedia in December. The VisualEditor lets editors create and modify real articles visually, using a new system where the articles they edit will look the same as when one reads them — like writing a document in a word processor.
The VisualEditor is now on 15 language Wikipedias

The VisualEditor is now on 15 language Wikipedias

Editors on fifteen Wikipedias – Arabic, Chinese, Dutch, English, French, German, Hebrew, Hindi, Italian, Japanese, Korean, Polish, Russian, Spanish and Swedish – can now get an idea of what the VisualEditor looks like in the “real world”, so they can give us feedback about how well it integrates with their current editing processes. We also want to get their thoughts on what aspects of development we should be prioritizing in the coming months.

The editor is still at an early stage and is missing significant functions, which we will address in the coming months. Because of this, we are mostly looking for feedback from experienced editors; the alpha VisualEditor is insufficient to really give new volunteers a proper experience of editing. We don’t want to promise an easier editing experience to new editors before it is ready.

As we develop improvements, we will push them live every two weeks to the wikis, allowing you to give us feedback as we go, and tell us what you want us to work on next.

How can I try it out?
The VisualEditor is now available to all logged-in accounts as a new preference, switched off by default, on the fifteen Wikipedias listed above. If you go to your “Preferences” screen and click into the “Editing” section, it will have an option labelled “Enable VisualEditor.”

Once enabled, for each article you can edit, you will get a second editor tab labelled “VisualEditor” next to the “Edit” tab. If you click this, after a little pause you will enter the VisualEditor. From here, you can play around, edit and save real articles and get an idea of what it will be like when complete.

At this early stage in our development, we recommend that after saving any edits, you check whether they broke anything. All edits made with the VisualEditor will show up in articles’ history tabs with a “VisualEditor” tag next to them, so you can track what is happening.

How can I help?
It’s vital that our software is available in the native language of as many of our volunteers as possible. If you speak one of these languages – or any of the other 280 languages that we support, like WelshPunjabiUrdu or Scots Gaelic - please consider looking at the translations and helping us improve them!

We would love your feedback on what we have done so far — whether it’s a problem you discovered, an aspect that you find confusing, the areas you think we should work on next, or anything else, please do let us know.

James ForresterProduct Manager, VisualEditor and Parsoid

(more…)

Try out the alpha version of the VisualEditor

Yesterday we launched an alpha, opt-in version of the VisualEditor to the English Wikipedia. This will let editors create and modify real articles visually, using a new system where the articles they edit will look the same as when you read them, and their changes show up as they type enter them — like writing a document in a word processor.

Why launch now?

We want our community of existing editors to get an idea of what the VisualEditor will look like in the “real world” and start to give us feedback about how well it integrates with how they edit right now. We also want to get their thoughts on what aspects should be priorities in the coming months.

The editor is at an early stage and is still missing significant functions, which we will address in the coming months. Because of this, we are mostly looking for feedback from experienced editors at this point, because the alpha VisualEditor is insufficient to really give them a proper experience of editing. We don’t want to promise an easier editing experience to new editors before it is ready.

As we develop improvements, they will be pushed every two weeks to the wikis, allowing you to give us feedback as we go, and tell us what you want us to work on next.

How can I try it out?

The VisualEditor is now available to all logged-in accounts on the English Wikipedia as a new preference, switched off by default. If you go to your “Preferences” screen and click into the “Editing” section, it will have an option labelled “Enable VisualEditor.”

Once enabled, for each article you can edit, you will get a second editor tab labelled “VisualEditor” next to the “Edit” tab. If you click this, after a little pause you will enter the VisualEditor. From here, you can play around, edit and save real articles and get an idea of what it will be like when complete.

At this early stage in our development, we recommend that after saving any edits, you check whether they broke anything. All edits made with the VisualEditor will show up in articles’ history tabs with a “VisualEditor” tag next to them, so you can track what is happening.

We would love your feedback on what we have done so far — whether it’s a problem you discovered, an aspect that you find confusing, what area you think we should work on next, or anything else, please do let us know.

James ForresterProduct Manager, VisualEditor and Parsoid

Inventing as we go: building a visual editor for MediaWiki

We at the Wikimedia Foundation, in conjunction with Wikia, are building the VisualEditor for Wikipedia and all other sites based on the MediaWiki software. This is taking us some time, and we often get asked what’s involved in this, just how hard it is, and why it’s taking us so long, so we thought we’d explain some of the intricacies in the most challenging software project the Wikimedia Foundation has ever worked on.

What are we trying to do?

We are creating software that will let users load, edit and save Wikipedia articles visually, bypassing the existing system that requires our users learn “wikitext,” a complex markup code. Instead, the articles they’re editing will look the same as when they’re reading them, and any changes they make will be obvious in their effects before they press save — just like writing a document in a word processor.

Haven’t people done this before?

Yes and no. There are lots of visual text editors out there, and even a few open source ones that can edit Web pages quite well, but they are insufficient for our needs for a number of reasons.

One criterion that is hugely important to us is that our editor should work with lots of languages. This is not just a matter of supporting certain right-to-left languages, or a few based on ideograms, but being able to use any and all of the 290 languages we currently provide. We also want to be able to do so seamlessly in the same documents to support our multi-lingual communities. Some of the languages we are committed to working with have very little software support, and we are often one of the few sources of written material for them, or at least, one of the largest.

Another issue is that wikitext has grown over the past 12 years to have a large number of rich and complicated features that are not just “simplified ways of writing HTML.” Though we originally intended many of these to be used only occasionally, they have often evolved to be at the core of how MediaWiki pages are now written by many Wikimedia users and more widely.

These advanced features include content transclusion (pulling in a “live” copy of one page or part of it into another), templates (transclusion with parts defined at the second page rather than the source), and parser functions (pages that show different things depending on hundreds of potential options, like the day of the week or whether another page exists). Attempting to retrofit this into an existing editor would have been exceptionally difficult, and more work than starting from scratch.

VisualEditor and Parsoid module stack

How VisualEditor and Parsoid work together (click to enlarge)

What is the parser?

Finally, we need to not just edit pages, but to read and save existing wiki pages in the old wikitext format, and continue to work with it in parallel to the new editor. We can’t throw away the huge amount of work our communities have done over the past 12 years, so we need to re-write the “translator.” This means making a two-way “parser” — a bit of software that converts wikitext into HTML and back again. Until now, we have only had a one-way parser; the second stage, converting back from what people want to write to wikitext, has had to be done by our editors in their heads.

This means that we’ve had to have an entirely separate project – the Parsoid – that can translate in both directions: from wikitext to Web pages and also back again. This is not remotely an easy task; you have to be very attentive in replacing a parser, as it’s hugely important that we don’t break anything. The old parser, and the wikitext “language” it interprets, just grew organically as people had ideas, and no one really designed it. There are nearly two billion versions of pages in Wikimedia’s wikis alone, and this lack of design means that there are a huge number of little rules we need Parsoid to follow to avoid “dirty-diffs” — issues where the wikitext would be broken by people using the VisualEditor.

Parsoid HTML+RDFa content model

Explaining the HTML+RDFa content model used by Parsoid (click to enlarge)

We use automated testing to “round-trip” 100,000 randomly chosen pages from the English Wikipedia (as a reasonable sample of wiki content in the Latin alphabet): taking the wikitext, converting it to HTML format and back to wikitext, and comparing the result. This helps us by identifying many issues to fix so that using Parsoid does not cause articles to break. Right now we get about 80 percent of these articles to round-trip without any differences at all (up from 65 percent in October), an additional 18 percent round-trip with only minor differences (whitespace, quote style etc.), and the remaining 2 percent of pages have differences that still need fixing (down from about 15 percent in October).

Learn more

As you can see, to make a visual editor that our users need is a large amount of work. No existing technologies could do what we wanted to, so we have needed to work very hard to make sure that we deliver this. We look forward over the next few months to showing off how editors will be able to use VisualEditor and Parsoid for a much better experience, free of learning any complicated codes, letting them focus on the content and not the tool they use to write it.

If you’re interested, we have a brief presentation about how Parsoid and VisualEditor work, and what they will look like on Wikipedia.

James ForresterProduct Manager, VisualEditor and Parsoid

Help us shape Wikimedia’s prototype visual editor

Today, the Wikimedia Foundation launched a new prototype “visual editor” for Wikimedia. The visual editor is a new editing environment that won’t require everyone to learn our special markup language in order to contribute to our projects.

Right now, if you try to edit the English Wikipedia’s article about the Wikimedia Foundation, or the Latin Wiktionary’s entry for “futūrus” (about to be), you get a lot of confusing characters interspersed amongst the recognisable text. Though it’s possible to learn what these mean and use them powerfully, many of our editors, and especially new editors, want to contribute content, not learn technical formatting.

We identified the difficulty in learning wikitext as a key inhibitor to growing our editor community in the Wikimedia movement’s strategic plan. We want the process of learning how to edit to be trivial, so our volunteers, both new and experienced, can devote themselves to what they edit. That’s why we’re building the visual editor, so that contributing to a wiki is as easy and natural as other modern editing systems, and new editors are not dissuaded from making their changes.

You may remember a similar announcement in December 2011, when we revealed a developer prototype of our “visual editor,” but after a great deal of feedback, we’ve reworked it so that it’s more useful to our community of users.

We learned a lot from building our first prototype. It was great how many of you helped with feedback, bug reports and comments about how we were doing. In the months since then, based on your feedback and technical issues we encountered, we’ve overhauled the entire editor. We changed the technical design and how it works, rewriting its components so that we can better support more editors. We’ve also integrated it into the MediaWiki platform, so now it can load and edit wiki articles, and not just sit separately.

A screenshot of the new visual editor

To build this iteration of our open-source visual editor, we have been working with some of the team from Wikia, a collaborative publisher that operates the largest network of video game, entertainment and lifestyle wikis in the world. We both believe that this kind of tool should be built not just for the Wikimedia wiki projects, but for everyone using MediaWiki software, and when it’s done we look forward to including the visual editor “out of the box” for anyone setting up a wiki with our software.

Thanks to all this, our new prototype is now live on mediawiki.org. This is just a demonstration, and very far from a finished product — for example, we haven’t yet added image or table handling. It’s currently locked down to only work on a self-contained area of the wiki, so that it doesn’t encounter any unsupported content or break anything else. We intend to work on small pieces of the overall story, releasing a new version every two weeks or so, and adding features one-by-one until the editor is good enough to deploy for everyone (and release in MediaWiki’s core).

Over the next few weeks and months, we will be working with the community — you — to find bugs, to focus on what our priorities should be, and most importantly, to make sure that what we’re building is right for you and that it supports your “workflow.”

So please, try out the prototype, see our frequently-asked questions and tell us what you think.

– The Visual Editor Team: Trevor Parscal, Inez Korczyński, James Forrester, Roan Kattouw, Rob Moen, Subramanya Sastry, Brion Vibber, Gabriel Wicke, Christian Williams.