Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Preparing for the migration from the Wikimedia Toolserver to Tool Labs

Last week-end, the Wikimedia Hackathon took place in Amsterdam, and we notably worked on the migration from the Wikimedia Toolserver to Tool Labs. According to the Roadmap, Tool Labs will have all the necessary features by the end of June 2013. As of then, tool maintainers will have one year to migrate their software. By the end of 2014, the Toolserver will be decommissioned.

What is working?

First of all: Tool maintainers do not have to care about virtual instances or other background stuff. You will develop on servers similar to the Toolserver, e.g. with the infrastructure for web services. The servers are running Ubuntu Precise. At the moment, there are replicas of six (out of seven) database clusters. The last one (with CentralAuth) is due in June. You can already work with all the big and many small Wikipedias, with Commons and Wikidata. You have access to all the data that is visible to registered users without special privileges in a wiki. You can create your own user databases. According to the first experiences, Tool Labs is fast.

In addition to home directories, you also have shared project storage. Tool Labs wants to make it as easy as possible to develop software together, which is why you can add others to your projects via the web interface. There is also a time travel feature! You can reset your files to the state of the last three hours, the last three days and the last two Sundays. The job system that is used in Tool Labs is OpenGridEngine. You can find an intro on the Tool Labs help page. Bugs can be reported in Wikimedia’s Bugzilla: Please use the product “Wikimedia Labs” and one of the components “Tools” or “Bots”. If you miss software in Tool Labs that could be of interest for others, too, please file a bug!

What about “Tools” versus “Bots”?

These are the names of the two projects Tool Labs consists of. The larger environment (Wikimedia Labs) is organized in projects, two of which form Tool Labs. They are an environment inside Labs that is customized for Toolserver users. The naming might be a bit misleading: The difference between “Tools” and “Bots” is not what you run in which project, but that you can run your tools in two different environments. The “Tools” project is a stable environment maintained by four admins (one of them a volunteer). There are no experiments with software versions here. In contrast to this, the “Bots” project is a more flexible environment in which you can play with changes in the environment itself, too. Here, it will be easier to get root access. (If you are interested, ask on the mailing list.)

Open tasks?

Apart from the open tasks on the roadmap, the documentation needs improvement. The pioneers among you can help others a lot by documenting experiences. Magnus Manske and Russell Blau have started to lead by example by adding a lot of documentation, and you can help as well! We are thinking about how to redirect deprecated links to migrated tools in the easiest way possible. The Tool Labs user interface also needs some love; feel free to come to us if you want to help here!

If you run into problems or have questions when migrating tools, be bold and ask! The best places are the labs-l mailing list or the IRC channel #wikimedia-labs connect. The admins’ nicks are Coren and petan. There is also a list of Frequently Asked Questions that you can expand. And finally: If you find that your tool needs more adaptation than you think you can manage on your own, talk to Johannes Kroll or myself at Wikimedia Deutschland for support!

Silke Meyer
Projektmanagerin für den Toolserver, Wikimedia Deutschland

One Response to “Preparing for the migration from the Wikimedia Toolserver to Tool Labs”

Leave a Reply