To keep up with the growth of Wikipedia and its community, one goal of the engineering team at the Wikimedia Foundation for this year is to establish a Quality Assurance (QA) practice for software development, including MediaWiki itself, extensions, and also projects like Article Feedback and Editor Engagement. But how do you establish a QA & Testing practice for a development process that involves so many contributors, with code coming in from so many sources and projects?
In software development, QA is often conflated with software testing, but testing is only a small part of QA in general. The goal of modern software testing is not only to discover defects, but also to investigate software in order to provide valuable information about that software from every point of view, from the user experience to how the software is designed. On the other hand, Quality Assurance is process work, examining the process by which the software is created, from design to code to test to release and beyond.
As QA Lead, I have created an example of what I believe to be the best available test automation “stack”, to pave the way and start the process of what I intend to be a reference implementation, an industry standard for high-quality browser test automation. We’re now hiring a QA Engineer whose primary responsibility will be to create and maintain browser-level test automation. In the course of creating those automated tests, we will be improving our use of the source code repository recently migrated from Subversion to git, we will be improving the beta labs test environment, and we will be expanding the use of our Continuous Integration in Jenkins.
But test automation isn’t everything, and we also have an opportunity to apply the Wikimedia community’s expertise in online volunteer collaboration to software quality. We’ve already started to explore this path with success: in May, we collaborated with Weekend Testing to validate the new frequent release schedule of MediaWiki to Wikimedia sites. Weekend Testing is an established global group of professional software testers who gather online every month for a different testing project, and testing Mediawiki versions on Wikipedia was a complex effort, executed well. In June, we collaborated with OpenHatch.org to test a near-final version of the new Article Feedback system that will be released to all of Wikipedia in the coming weeks. OpenHatch is an organization
dedicated to matching interested participants in open source software to projects that need participants. This was the first testing project for OpenHatch, and it went well; Article Feedback is much improved because of it.
We are now hiring a Volunteer QA Coordinator, who will be working to create a culture of quality testing and investigation of software related to Wikimedia, both within the Wikimedia community itself, and in collaboration with the greater software testing culture. And we are already planning future activities with both Weekend Testing and OpenHatch.
My first few months as QA Lead at the Wikimedia Foundation have been devoted to creating an environment where the QA Engineer and the Volunteer QA Coordinator will thrive. I am really looking forward to collaborating with the talented people we will hire for these roles. My own role will be shifting as these new practices start to take hold. I will be looking to the future, to bring in innovative and creative approaches to software QA and testing of the highest possible quality.
QA Lead Engineer