Earlier this month, about thirty MediaWiki developers and interested technologists gathered in New Orleans to learn and to work on Wikimedia’s technical infrastructure.  We made broad progress on the infrastructure of innovation at Wikimedia (notes).  Specifically:

• We are now much closer to officially opening the doors to Wikimedia Labs and giving far more people the ability to contribute to MediaWiki without having to set up and maintain their own development environments at home.  Wikimedia Labs will provide hosted, virtualized test and development sandboxes for new and experienced programmers and systems administrators.  Many developers got beta Labs accounts, we tested at a larger scale, and we fixed several bugs.
• Developers agreed to create a file backend abstraction layer to enable large-scale MediaWiki installations to use one of several storage systems to contain big collections of big media files.  (Wikimedia plans on using Swift, which is open source.) Microsoft’s Ben Lobaugh and SAIC’s DJ Bauch collaborated towards improving MediaWiki’s performance on Microsoft technologies as well.  Developers made architectural decisions, refactored some existing code, and improved documentation and tests for the SwiftMedia extension to MediaWiki.
• 

  Chad Horohoe teaching developers unit testing

  We now have a continuous integration server up and running.  This will continuously run tests checking on the latest new features and bugfixes that developers write, resulting in fewer bugs and faster development. Developers will need to write tests to reap the benefits, so Chad Horohoe taught a test-writing workshop.

• Max Semenik finished and demonstrated the first version of his API Query Sandbox.  This allows software developers anywhere to experiment with ways to automatically get data from Wikipedia or other sites that run MediaWiki, thus enabling wider and deeper reuse of Wikimedia content.
• Operations folks continued the Puppetization of our infrastructure: they completely reworked Varnish management in Puppet, and worked on Puppet configurations for SwiftMedia testing. This configuration management work will ensure that ops can move faster and more confidently in building and maintaining Wikimedia infrastructure. And Canonical’s Mark Mims and Kapil Thangavelu worked on improving methods for Wikimedia developers “to spin up stacks of services within the labs environment” using Juju (more details).
• 

  Brion Vibber leading developers into the "glorious Git future"

  Since the engineering department is planning a switch from Subversion to Git in the next few months, Brion taught nearly everyone there how Git works (slides, audio), and how we’ll be using Git in the future. This change in our source code repository and workflow will, we hope, enable more speed and flexibility in development, both for WMF developers and community contributors.
• We prioritized and addressed several open requests for the operations team and defect reports about the latest version of MediaWiki, 1.18, which had just been deployed across WMF sites.
• Roan found and fixed an issue that was spouting symbolic link errors into our Apache logs, so now it’ll be easier for us to see more dangerous errors in those logs.
• Google Summer of Code students Salvatore Ingala and Kevin Brown made progress on integrating their summers’ work into MediaWiki as used and deployed by others; Salvatore and WMF developer Roan Kattouw have a plan for getting his user scripts improvements reviewed and deployed, so they can benefit Wikimedia readers and editors.
• A volunteer came in on Friday night knowing nothing about developing for MediaWiki, and by the end of the weekend had a working development environment on her laptop and had some ideas about how to contribute.
• We had substantive conversations about the summer internship program and about third-party collaboration that will affect how we work in the future.

We also ate dinner together, walked Bourbon Street, and generally got to know colleagues we’d never met before.  I expect these relationships will bear fruit for years to come.

Thanks to Ryan Lane and Dana Isokawa for organizing the event with me, and thanks to Launch Pad New Orleans for providing the venue!

Our next developers’ event is a hackathon in Mumbai November 18-20 concentrating on internationalization, localization, and mobile work.  To find out about other upcoming Wikimedia technical events, check the meetings wiki page, and follow @MediaWikiMeet on Identi.ca or Twitter.

Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

Every year, Wikipedia usage goes upward, and every year the technical folks working and volunteering with Wikimedia have to plan, purchase, and implement new servers to keep up to the growing popularity of Wikipedia and its sister projects.  With the advances in computing, running 9 new application servers this year took the load of 36 application servers from 3 years ago.

So when we upgrade, what happens to the old equipment that is too slow for Wikipedia, but not too slow for MANY other non-profits?  We donate them!  These systems were 1U rackmount servers, dual cpu 2.5-3, single core, 2-4GB of RAM, and 2-4 HDD Bays with 1-2 80-250GB HDDs. This year, we have  three non-profits who received our older systems (in alphabetical order): Drupal.org, OpenStreetMap Foundation, and Sugar Labs.

Drupal.org

Drupal is a free software package that allows an individual or a community of users to easily publish, manage and organize a wide variety of content on a website. Tens of thousands of people and organizations are using Drupal to power scores of different web sites.

OpenStreetMap Foundation

The OpenStreetMap Foundation is an international non-profit organisation supporting but not controlling the project. It is dedicated to encouraging the growth, development and distribution of free geospatial data and to providing geospatial data for anybody to use and share.

OpenStreetMap is an open initiative to create and provide free geographic data such as street maps to anyone who wants them.

Sugar Labs

The mission of Sugar Labs^® is to produce, distribute, and support the use of the Sugar learning platform; it is a support base and gathering place for the community of educators and developers to create, extend, teach, and learn with the Sugar learning platform.

We hope the recipients of our servers will be able to put them to good use!

Below are some common questions involving Wikimedia and the server donation process:

Q. How can I get some of the decommissioned donation servers?

A. The best place to follow the goings on of our technical team is here, on the Wikimedia Technical Blog.  When we have a batch of servers up for decommissioning and donation, we will announce it on the tech blog, and instructions on how to apply to receive some servers.

Q. Who is eligible to apply for servers?

A. We try to only donate servers to other non-profits whose core values are similar or in support of our own.  This means we do not donate them for individual use.   Since these servers were purchased with donations to support Wikimedia, we feel we need to further donate them to other like-minded organizations, since that is how the money for the servers was meant to be spent.

Q. How often does this happen?

A. Most servers are kept in use by Wikimedia beyond three years.  Many of our servers that we have turned off in this batch are anywhere from 3 to 5 years old.  We only replace them when it makes sense from the technical standpoint to do so.  This means we cannot just say ‘we will do this every X months.’  We try to get the most use out of every server, as they were donated or purchased with donations.  So there is no set date, just keep checking the Wikimedia Technical Blog, when we have more to donate, we will say so there!

Q. I am a student/person/so and so, and I want to learn to develop and do such and such.  Can you send me a server?

A. Sorry, unfortunately it is just not realistic or fair of us to try to sort out which personal use requests for servers are legitimate and which are folks wanting computers for any other reason.  We choose to limit our donations to other like minded non-profit organizations.

Rob Halsell
Systems Administrator

Just to let folks know, we have had quite a large interest in our donation of some of our decommissioned servers.  In fact, I have way too many emails!

So to be fair, rather than just stop today, we will stop accepting submissions for this next Monday, September 28th.  That means if you want your proposal/request in the running, you have to have it emailed to servers@wikimedia.org by Midnight GMT this coming Sunday, Sept. 27th.

For ease of reference, here is a copy of the post from the start of this process:

It is that time again.  We have approx 35 servers to donate to a good home.  These are servers that Wikimedia has used on the projects for 3+ years, so they are out of warranty and just not fast enough for us to keep using on the cluster.

The servers will go out to homes for folks who are willing to pay for the freight.  They are as follows:

• Dual CPU 2.5 GHz AMD
• 3-4GB RAM Each
• Most have 80 GB or larger HDD

Disclaimers: The Wikimedia Foundation does not guarantee the operation or use of these servers in any shape or form.  They are old, some may have dying fans, bad hdd sectors, and the like.  Servers have been wiped of information, and they ran through that, but no promises on function!

If you would like to receive some of these servers for your NONPROFIT use, please email servers@wikimedia.org.  Please include in your email how you will be using the servers, and the address they would be shipped to.  We will review all requests and try to fairly pick out where they go.  (Selection process may be refined, but it also may just include throwing darts at a board to break up ties.)

Additions: Due to request, the servers are indeed located in Tampa, FL USA.  Zip code 33602 for shipping purposes.  This means that if you are international, shipping this hardware is really not cost effective for you.  If you want to be in the running still, and are comfortable with personally handing all customs, duties, export, and tax issues, go ahead and email us.

Rob Halsell, Operations Engineer

Correction: Dates were off.

It is that time again.  We have approx 35 servers to donate to a good home.  These are servers that Wikimedia has used on the projects for 3+ years, so they are out of warranty and just not fast enough for us to keep using on the cluster.

The servers will go out to homes for folks who are willing to pay for the freight.  They are as follows:

• Dual CPU 2.5 GHz AMD
• 3-4GB RAM Each
• Most have 80 GB or larger HDD

Disclaimers: The Wikimedia Foundation does not guarantee the operation or use of these servers in any shape or form.  They are old, some may have dying fans, bad hdd sectors, and the like.  Servers have been wiped of information, and they ran through that, but no promises on function!

If you would like to receive some of these servers for your NONPROFIT use, please email servers@wikimedia.org.  Please include in your email how you will be using the servers, and the address they would be shipped to.  We will review all requests and try to fairly pick out where they go.  (Selection process may be refined, but it also may just include throwing darts at a board to break up ties.)

Additions: Due to request, the servers are indeed located in Tampa, FL USA.  Zip code 33602 for shipping purposes.

Rob Halsell, Operations Engineer

In working on the servers, some apache config files were made inoperable.  This is on a misc. services computer named Singer.  This is the host for our blogs, as well as some other web-facing info.  As such, the cached blogs are affected, but not the tech blog.  (It was, but it was the easiest to get back online.)

Apologies for any annoyance this single server downtime may have caused anyone.  Rest assured, it will be fixed and steps will be taken to prevent it from occurring in the future.

The Wikimedia projects have been running on the same commodity hardware for many years now and every now and then we decide to decommission some of our older machines. This not only allows us to free up space for new servers but also lets us use more energy efficient hardware.

While searching around for a new home for our old but still very useful servers we came across two linked organizations: northxsouth & San Francisco Community Collocation Project (SFCCP). Both of these groups help out their local and regional communities by using open source software to better spread information within various media spaces.

The SFCCP is active within the San Francisco community while northxsouth works with various Latin American countries.

Our donation of servers was happily received and I’m excited to report that they will soon be humming along and serving the public for a long time to come.

Tomasz Finc, Software Developer