Earlier this month, about thirty MediaWiki developers and interested technologists gathered in New Orleans to learn and to work on Wikimedia’s technical infrastructure.  We made broad progress on the infrastructure of innovation at Wikimedia (notes).  Specifically:

• We are now much closer to officially opening the doors to Wikimedia Labs and giving far more people the ability to contribute to MediaWiki without having to set up and maintain their own development environments at home.  Wikimedia Labs will provide hosted, virtualized test and development sandboxes for new and experienced programmers and systems administrators.  Many developers got beta Labs accounts, we tested at a larger scale, and we fixed several bugs.
• Developers agreed to create a file backend abstraction layer to enable large-scale MediaWiki installations to use one of several storage systems to contain big collections of big media files.  (Wikimedia plans on using Swift, which is open source.) Microsoft’s Ben Lobaugh and SAIC’s DJ Bauch collaborated towards improving MediaWiki’s performance on Microsoft technologies as well.  Developers made architectural decisions, refactored some existing code, and improved documentation and tests for the SwiftMedia extension to MediaWiki.
• 

  Chad Horohoe teaching developers unit testing

  We now have a continuous integration server up and running.  This will continuously run tests checking on the latest new features and bugfixes that developers write, resulting in fewer bugs and faster development. Developers will need to write tests to reap the benefits, so Chad Horohoe taught a test-writing workshop.

• Max Semenik finished and demonstrated the first version of his API Query Sandbox.  This allows software developers anywhere to experiment with ways to automatically get data from Wikipedia or other sites that run MediaWiki, thus enabling wider and deeper reuse of Wikimedia content.
• Operations folks continued the Puppetization of our infrastructure: they completely reworked Varnish management in Puppet, and worked on Puppet configurations for SwiftMedia testing. This configuration management work will ensure that ops can move faster and more confidently in building and maintaining Wikimedia infrastructure. And Canonical’s Mark Mims and Kapil Thangavelu worked on improving methods for Wikimedia developers “to spin up stacks of services within the labs environment” using Juju (more details).
• 

  Brion Vibber leading developers into the "glorious Git future"

  Since the engineering department is planning a switch from Subversion to Git in the next few months, Brion taught nearly everyone there how Git works (slides, audio), and how we’ll be using Git in the future. This change in our source code repository and workflow will, we hope, enable more speed and flexibility in development, both for WMF developers and community contributors.
• We prioritized and addressed several open requests for the operations team and defect reports about the latest version of MediaWiki, 1.18, which had just been deployed across WMF sites.
• Roan found and fixed an issue that was spouting symbolic link errors into our Apache logs, so now it’ll be easier for us to see more dangerous errors in those logs.
• Google Summer of Code students Salvatore Ingala and Kevin Brown made progress on integrating their summers’ work into MediaWiki as used and deployed by others; Salvatore and WMF developer Roan Kattouw have a plan for getting his user scripts improvements reviewed and deployed, so they can benefit Wikimedia readers and editors.
• A volunteer came in on Friday night knowing nothing about developing for MediaWiki, and by the end of the weekend had a working development environment on her laptop and had some ideas about how to contribute.
• We had substantive conversations about the summer internship program and about third-party collaboration that will affect how we work in the future.

We also ate dinner together, walked Bourbon Street, and generally got to know colleagues we’d never met before.  I expect these relationships will bear fruit for years to come.

Thanks to Ryan Lane and Dana Isokawa for organizing the event with me, and thanks to Launch Pad New Orleans for providing the venue!

Our next developers’ event is a hackathon in Mumbai November 18-20 concentrating on internationalization, localization, and mobile work.  To find out about other upcoming Wikimedia technical events, check the meetings wiki page, and follow @MediaWikiMeet on Identi.ca or Twitter.

Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

Congratulations to the seven Google Summer of Code students who made it through the summer of 2011! They all accomplished a great deal, but want to continue contributing to ensure their work maximally benefits Wikimedia.

Yuvi Panda‘s assessment parsing/aggregating extension aims “to make it easier to select and export article selections for various offline collections.” Yuvi needs some code review and suggestions on how to improve it to meet the Foundation’s quality standards for deployability, as he wrote the developers’ mailing list.

Salvatore Ingala worked on making gadgets customizable. As he elaborated, that means:

• “allowing gadgets to easily declare the list of configuration
  variables they have;
• allowing users to easily change those settings, with an easy-to-use
  UI integrated to the Special:Preferences page.”

The next step is merging his code into trunk, which Salvatore’s planning with other MediaWiki developers.

Kevin Brown created the ArchiveLinks project to address the problem of linkrot on Wikipedia:

In articles we often cite or link to external URLs, but anything could happen to content on other sites — if they move, change, or simply vanish, the value of the citation is lost. ArchiveLinks rewrites external links in Wikipedia articles, so there is a ‘[cached]‘ link immediately afterwards which points to the web archiving service of your choice. This can even preserve the exact time that the link was added, so for sites which archive multiple versions of content (such as the Internet Archive) it will even link to a copy of the page that was made around the time the article was written.

Kevin’s next step: getting a security review of his code, getting a starter feed set up so that the Internet Archive can start archiving it, and campaigning to interest Wikimedians and thus eventually get consensus to turn it on. At least one Wikimedian has already praised Kevin for his work.

Akshay Agarwal wrote a MediaWiki extension, SignupAPI, that makes it easier for a new user to create an account. “This extension creates a special page that cleans up SpecialUserLogin from signup related stuff, adds an API for signup, adds sourcetracking for account creation & provides Ajax-ified validation for signup form.” Akshay’s waiting for code review and discussion before the project can move forward further and benefit Wikimedia users.

Yuvi, Salvatore, Kevin, and Akshay all worked on features that they aim to get into Wikimedia Foundation-run wikis, such as Wikipedia, Wikisource, Wikinews, etc., sooner rather than later. In contrast, three students worked on extensions that will primarily benefit the larger MediaWiki community. For example, Yevhenii Vlasenko‘s project was a “UserStatus” feature for SocialProfile. The SocialProfile extension is not currently deployed on any WMF wikis, but will benefit several other MediaWiki administrators and users. Zhenya finished his work but would like to continue by integrating better with social networks.

And two students worked on Semantic MediaWiki, which is also not currently deployed on any Wikimedia Foundation sites. Devayon Das made a “QueryCreator” and other improvements, and hopes to simplify its layout, make its interface easier to use, and add some features. And Ankit Garg worked on “Semantic Schemas”.

Congratulations to the students and their mentors.  Here’s hoping they’re all here to help out when next year’s interns roll in! :-)  And I’m looking forward to meeting Kevin and Salvatore, and introducing them to other Wikimedia & MediaWiki developers, at the New Orleans developers’ meetup next month.

Sumana Harihareswara
Volunteer Development Coordinator
Wikimedia Foundation

MediaWiki’s Google Summer of Code students have been busy! We’re more than halfway through the summer, so here’s what they’re up to:

• Akshay Agarwal’s “Account Creation, Login Screens and AJAX-ification of everything” (mentor: Brandon Harris). Code, project status.
  The last task I accomplished: “Added source tracking functionality in the account creation API that I am building.”
  Something I’ve learned: “True learning can happen only in an open environment & with a highly supportive community.”
• Kevin Brown’s “Working Archival for Web References/Citations,” “to facilitate the archival of external links used as references in the English Wikipedia” (mentor: Neil Kandalgaonkar). Code, project notes.
  The last task I accomplished: “Adding support for wget local archival, currently working on feed for external archival services.”
  Something I’ve learned: “Where do I start? A lot. I think the biggest thing is probably managing a large project and time management, which I still have a lot to learn on.”
• Devayon Das’s “Improving Semantic Search/Semantic Query usability issues in SMW” (mentor: Markus Krötzsch). Code, project notes.
  The last task I accomplished: “Added RSS links to the results generated by the Query Creator interface I’m building.”
  Something I’ve learned: “A 30 second chat with a community member can save you 30 minutes of scratching your head in frustration.”
• Ankit Garg’s “Semantic Schemas extension” (mentor: Yaron Koren). Code.
  The last task I accomplished: “I finished adding the inheritance support to the PageSchema XML structure.”
  Something I’ve learned: “I have a learned a great deal of PHP; also how to manage a huge project.”
• 

  "A 30 second chat with a community member can save you 30 minutes of scratching your head in frustration."

  
  Salvatore Ingala’s “AMICUS: Awesome Monolithic Infrastructure for Customization of User Scripts” (mentors: Max Semenik and Brion Vibber). Code, project notes.
  The last task I accomplished: “I made a prototypal user interface for editing preferences of an existing gadget, HotCat.”
  Something I’ve learned: “Unit testing is boooooring, but ends up saving you a lot of time!”
• Yuvi Panda’s “Making Offline Wikipedia Article Selection Easier with Mediawiki Extensions” (mentor: Arthur Richards). Code, project.
  The last task I accomplished: “Filter articles based on name, quality and importance.”
  Something I’ve learned: “That spending time talking to everyone involved in the process from start to finish (devs, community maintainers, etc.) saves a truckload of time later on.”
• Zhenya Vlasyenko’s “MediaWiki Extension: SocialProfile – UserStatus feature” (mentor: Jack Phoenix). Code.
  The last task I accomplished: “Internalization of the UserStatus feature with the help of the MakeGlobalVariablesScript hook.”
  Something I’ve learned: “I’ve found out for myself a new ways of data interaction between PHP and Javascript… Convinced that knowing some tricks and hooks can greatly save time.”

Aigerim Karabekova, who was working on extension release management, ran into several delays (including medical issues) and the project has been dropped. We’re glad she made the attempt and wish her the best.

Continued best wishes to Zhenya, Yuvi, Salvatore, Ankit, Devayon, Kevin, and Akshay as they work to make MediaWiki, and the Wikimedia experience, better.  We’re glad to be helping young developers learn how to contribute to our community.

Sumana Harihareswara
Wikimedia Foundation, Volunteer Development Coordinator

We received more than 25 proposals for this year’s Google Summer of Code, and several mentors put many hours into evaluating project ideas, discussing them with applicants, and making the tough decisions.  Our final choices, the Google Summer of Code students for MediaWiki for 2011:

• Akshay Agarwal‘s “Account Creation, Login Screens and AJAX-ification of everything” (mentor: Brandon Harris)
• Kevin Brown’s “Working Archival for Web References/Citations,” “to facilitate the archival of external links used as references in the English Wikipedia” (mentor: Neil Kandalgaonkar)
• Devayon Das‘s “Improving Semantic Search/Semantic Query usability issues in SMW” (mentor: Markus Krötzsch)
• Ankit Garg‘s “Semantic Schemas extension” (mentor: Yaron Koren)
• Salvatore Ingala‘s “AMICUS: Awesome Monolithic Infrastructure for Customization of User Scripts” (mentors: Brion Vibber and Max Semenik)
• Aigerim Karabekova‘s “Extension Release Management” (mentors: Sam Reed, Priyanka Dhanda, and Chad Horohoe)
• Yuvi Panda‘s “Making Offline Wikipedia Article Selection Easier with Mediawiki Extensions” (mentor: Arthur Richards)
• Zhenya Vlasyenko‘s “MediaWiki Extension: SocialProfile – UserStatus feature” (mentor: Jack Phoenix)

You’ll be hearing more about each of these projects in the next few weeks!

Congratulations to this year’s students, and thanks to all the applicants, as well as MediaWiki’s many mentors, developers who evaluated applications, and Google’s Open Source Programs Office.  The accepted students now have a month to ramp up on MediaWiki’s processes and get to know their mentors (the Community Bonding Period) and will start coding their summer projects on or before May 23rd.  As organizational administrator for MediaWiki’s GSoC participation, I’ll be keeping an eye on all eight students and helping them out.

Good luck!

For the sixth year in a row, Wikimedia is participating in the Google Summer of Code program. Google Summer of Code (GSoC) is a program where Google pays summer students USD 5000 each to hack open source projects during the summer (read more).

Over time, MediaWiki has benefited from GSoC students and their projects. For example, Samuel Lampa’s 2010 RDF import/export extension in Semantic MediaWiki is in use. And Jeroen De Dauw, GSoC student in 2009 and 2010, is now a persistently contributing member of the MediaWiki community, as is Brian Wolff, 2010 GSoC student.

In the past, the administrative and management challenges of GSoC have been an extra task that take engineers’ time, and too often fell through the cracks. So this year, Rob Lanphier asked me to act as organizational administrator for MediaWiki’s involvement, via the Wikimedia Foundation.

I’m recruiting students to apply, getting project ideas, and managing the application process overall. Once we choose the students and they start ramping up and working, I will also help mentors manage their students and keep communication going, to make sure that every GSoC student’s project gets delivered and gets used!

We hope 2011′s students will develop useful chunks of MediaWiki (core, extensions, gadgets, scripts, or utilities), help us get their code shipped, and stay in the MediaWiki community afterwards.

This year’s ideas include writing and implementing cite templates in a PHP extension, improving the ImageTagging extension, XML dump work, pre-commit checks in our code repositories, and more. And of course we want to hear your own ideas, too! Interested?

University, community college, and graduate students around the world are eligible to apply to Google Summer of Code. You don’t need to be a computer science or IT major, and you can work from home.

We are looking for students who already know PHP. It’s also great if you have some experience with LAMP, MAMP, LAPP, or one of those kinds of stacks, and with the Subversion version control system. If you haven’t contributed to MediaWiki before, How to become a MediaWiki hacker is a good place to start.

If you’d like to participate, check out the timeline. Make sure you are available full-time from 23 May till 22 August this summer, and have a little free time from 25 April till 23 May for ramp-up.

If you’re interested, please sign up on our wiki page and start talking with us on IRC in #mediawiki on Freenode about a possible project! Then you can submit your proposal via the official GSoC website. The deadline for you to submit a project proposal is April 8th, but we encourage you to start early and talk with us about your idea first.

And, to repeat what Brion once said:

If you’re an experienced MediaWiki developer and would like to help out with selecting and mentoring student projects, please give us a shout! We’ll take you even if you live in the southern hemisphere. ;) We need folks who’ll be available online fairly regularly over the summer and are knowledgeable about MediaWiki — not necessarily knowing every piece of it, but knowing where to look so you can help the students help themselves.

We’re looking forward to hacking with you!

Sumana Harihareswara
MediaWiki Coordinator, GSoC 2011

• Extension management platform - Creating an awesome extension management platform for MediaWiki, facilitating the installation, updating, removal and configuration of extensions.  Student: Jeroen De Dauw, Mentor: Brion Vibber
• Improve metadata support - Improve metadata support for uploaded media in MediaWiki by displaying embedded IPTC and XMP metadata.  Student: Brian Wolff, Mentor: Chad Horohoe
• General RDF export/import in Semantic MediaWiki - Extend the import/export functionality of Semantic MediaWiki (SMW) to allow also full, general RDF import.  Student: Samuel Lampa, Mentor: Denny Vrandecic.
• Javascript overhaul of Semantic MediaWiki – Improve and extend the Javascript for Semantic MediaWiki and some of its spinoff extensions, most notably Semantic Forms.  Student: Sanyam Goyal, Mentor: Yaron Koren
• Wikisource Legal Tool - Creating a tool to format judicial decisions, legal scholarship, and statutes for Wikisource.  Student: Stephen LaPorte  Mentor: Ariel Glenn
• Reasonably efficient interwiki template transclusion – allow MediaWiki users to insert (transclude) templates from a wiki to another on Wikimedia Foundation (WMF) wikis (Wikipedia, Wikimedia Commons, etc.).  Student: Peter Potrowl, Mentor: Roan Kattouw

More detailed information on all of these projects can be found on our GSoC 2010 projects page.  Also, Wikipedia Signpost is highlighting this work over the coming weeks, starting with a summary of Brian Wolff’s XMP metadata project.

Though not all projects were finished completely as specified, all were completed to a sufficient degree that we felt very comfortable passing all of the students, and all of the students produced code we’re very happy to have.  Note that there is no guarantee that anything here will get beyond the proof-of-concept stage.  However, we’re hopeful that much of this work will find broader adoption, and we’re looking forward to that.

We hope that all of the students stick around as MediaWiki contributors long after the summer is over.  Please join us in thanking them for their participation this year!

Once again in 2010, Wikimedia Foundation is participating in Google Summer of Code.  I’m happy to announce that we’ve selected six students to participate this summer:

• Extension management platform
  Student: Jeroen De Dauw
  Mentor: Brion Vibber
  Goal: Creating an awesome extension management platform for MediaWiki, facilitating the installation, updating, removal and configuration of extensions. (student application)
• Improve metadata support
  Student: Brian Wolff
  Mentor: Chad Horohoe
  Goal: Improve metadata support for uploaded media in mediawiki by displaying embedded IPTC and XMP metadata (student application)
• General RDF export/import in Semantic MediaWiki
  Student: Samuel Lampa
  Mentor: Denny Vrandecic
  Goal: Extend the import/export functionality of Semantic MediaWiki (SMW) to allow also full, general RDF import. (student application)
• Javascript overhaul of Semantic MediaWiki
  Student: Sanyam Goyal
  Mentor: Yaron Koren
  Goal: Improve and extend the Javascript for Semantic MediaWiki and some of its spinoff extensions, most notably Semantic Forms – this would include transferring over much of the Javascript to use the jQuery library, which is now becoming a MediaWiki standard. (student application)
• Wikisource Legal Tool
  Student: Stephen LaPorte
  Mentor: Ariel Glenn
  Goal: Creating a tool to format judicial decisions, legal scholarship, and statutes for Wikisource. (student application)
• Reasonably efficient interwiki template transclusion
  Student: Peter Potrowl
  Mentor: Roan Kattouw
  Goal: The aim is to allow MediaWiki users to insert (transclude) templates from a wiki to another on WikiMedia Foundation (WMF) wikis (Wikipedia, Wikimedia Commons, etc.). (student application)

We had an exceptional set of really great proposals this year, and an engaged mentor group helping with the selection process. It was both wonderful to have so many choices, and really sad that we couldn’t pick them all, but in the end, we had to narrow the list down. Our six slots represent 100% growth from previous years’ Summer of Code engagements and that’s a pretty exciting stretch.

To the students that weren’t selected: do know that we were inspired by the quality level of all of the proposals, and we had to turn down some really exceptional proposals. Please don’t be discouraged, and do consider us next year!

To the students selected: congratulations! Welcome aboard! We really look forward to working with you to make sure you are successful and have a great time in the process.

To everyone volunteering as a mentor who helped with the selection process: thank you for your effort and dedication! There was a lot to sort through, but I think we can all feel great that we have a group of very capable students on the case this year thanks to your work.

More detailed info available here.

Towards the Fun!

I’m very excited to announce that LiquidThreads, the next-generation discussion system that I’ve spent the last few months developing for the Wikimedia Foundation, is now in beta testing on liquidthreads.labs.wikimedia.org.

(more…)

Reposting the announce from Roan’s wikitech-l mailing list post:

Yesterday, the selection of GSoC projects was officially announced. For MediaWiki, the following projects have been accepted:

• Niklas Laxström (Nikerabbit), mentored by Siebrand, will be working on improving localization and internationalization in MediaWiki, as well as improving the Translate extension used on translatewiki.net
• Zhe Wu, mentored by Aryeh Gregor (Simetrical), will be building a thumbnailing daemon, so image manipulation won’t have to happen on the Apache servers any more
• Jeroen de Dauw, mentored by Yaron Koren, will be improving the Semantic Layers extension and merging it into the Semantic Google Maps extension
• Gerardo Antonio Cabero, mentored by Michael Dale (mdale), will be improving the Cortado applet for video playback (I’m a bit fuzzy on the details for this one)

The official list with links to (parts of) the proposals can be found at the Google website; lists for other organizations can be reached through the list of participating organizations.

The next event on the GSoC timeline is the community bonding period, during which the students are supposed to get to know their mentors and the community. This period lasts until May 23rd, when the students actually begin coding.

Starting now and continuing at least until the end of GSoC in August, you will probably see and hear from the students on IRC and the mailing lists and hear about the projects they’re working on. To repeat the crux of an earlier thread on this list: be nice to these special newcomers, make them feel welcome and comfortable, and try not to bite them :)

To the mentors and students: have fun!

Roan Kattouw (Catrope)

Google Summer of Code is now open for student applications!

We’ve had 5 submissions come in so far… don’t be shy! :) Also don’t be shy about hanging out on our mailing lists and IRC channels and getting feedback from other MediaWiki developers on your project ideas.

The more feedback you get, the better you can make your submission… and the awesomer the result will be!

The application period ends April 3, 2009 at 19:00 UTC — don’t be late!

(We could still use a couple more project mentors too…)