Editor retention: Editing tools
In July, the VisualEditor team began switching the deployment from opt-in alpha to opt-out beta, so becoming the default editor for users of the various Wikipedias. The deployed version of the code was updated three times (1.22-wmf10
), with several mid-deployment releases as the code was developed to patch urgent issues. There were a number of user interface improvements, most notably to the references insertion dialog, alongside fixes to a number of bugs uncovered by the community.
In July, the Parsoid team supported the deployment of VisualEditor as default editor on eight Wikipedias, continuing to monitor bug reports, feedback pages, and village pump and fixed a number of bugs to eliminate instances of dirty diffs and other corruption that were reported. An absence of performance issues let us focus our attention on functionality and dirty-diff related bugs. This continued to be the primary focus of our work this month. On the staffing side, C. Scott Ananian joined the Parsoid team as a full-time employee — he has been working with us since earlier this year, first as a volunteer and then as a contractor. Marc Ordinas i Llopis from Spain and Arlo Breault from Canada joined the Parsoid team as contractors this month.
Editor engagement features
This month, we released two new prototypes
to showcase some ideas around Flow-enabled user-to-user discussion. We continued to collect user feedback and prioritize use-cases for a potential minimum viable product.
In July, we released our final features for Notifications on the English Wikipedia and mediawiki.org and meta.wikimedia.org. Benny Situ completed development of HTML Email notifications
, as well as improved notifications
, based on designs by Vibha Bamba. Fabrice Florin managed the release of these final features, and prepared this release plan
to deploy Notifications on more wiki projects, starting with French and Polish Wikipedias in August. Dario Taraborelli and Matthias Mullie updated our new metrics dashboards
, while Aaron Halfaker completed his report on our A/B test of new user activity
. To learn more, visit the project portal
, read the FAQ page
and join the discussion on the talk page
In July, we deployed a few last features and bug fixes for the Article Feedback Tool
(AFT5) on the English
Wikipedias. Matthias Mullie released the auto-archive
feature, as well as this list of articles with feedback enabled on enwiki
and on frwiki
. At the request of the French Wikipedia community, he also developed new feedback notifications
to let users know when feedback is marked as useful for a page they watch (or for a comment they posted). The team plans to make the AFT5 tool available to other wiki projects interested in testing this tool, provided that no new development is required to support their needs, as outlined in the release plan
Editor engagement experiments
Editor engagement experiments
In July, the Editor Engagement Experiments (E3) team made progress on a number of continuing projects. In terms of features, the team also completed work to integrate the onboarding new Wikipedians
project with new infrastructural changes and feature releases.For the GettingStarted
, E3 collaborated with Platform engineering to ensure compatibility with the new “SUL2
” cross-wiki authentication architecture. For the GuidedTour extension
, the team completed a first release of support for guided tours of the VisualEditor
interface, alongside tours of the legacy wikitext editor, and developed a plan to refactor
the GuidedTour extension as well as its API. E3 also planned for its sixth A/B test of the GettingStarted workflow (see proposed specification and mockups
). As an addition to the team’s redesign of account creation and login (launched in May-June), we enhanced the design of the form for users who fulfill account creation requests for others.E3 team member Matthew Flaschen also worked with two Google Summer of Code
students on their projects. Richa Jain
is working on the Annotator extension
, which allows adding inline comments to a wiki page. Rahul Maliakkal
is working on the Pronunciation Recording extension
, for adding audio of pronunciations to Wiktionary.
On the experimental tools and data analysis front, E3 completed a significant rewrite of the Puppet configuration for EventLogging, our data collection pipeline, among other changes. For the MediaWiki-Vagrant portable desktop development environment, E3 added support for flexibly provisioning and unit testing extensions such as GettingStarted, GuidedTour, ParserFunctions, EventLogging, and others. Last but not least, the micro-survey of gender of new account registrations was enabled on German, French, Italian, and Polish Wikipedias, while data analysis on the English Wikipedia results began.
2013 Wikimedia fundraiser
In July, the fundraising team did its first successful tests of our new payments gateway: Adyen. The (as yet) US-only Credit Card backup gateway performed similarly to our primary credit card processor in A/B testing, and can be successfully used as a failover. We also ran, for the first time, several short campaign tests targeted at mobile devices in the US. In these tests, users were able to choose between Paypal or Amazon Payments. Additional tests to determine peak times, appropriate localities, and optimum messaging for mobile campaigns will continue throughout August, as the campaigns are prepared.
In July 2013, MediaWiki 1.22wmf10 through 1.22wmf13 were successfully deployed to Wikimedia project sites. We skipped the week of July 4th as there was reduced capacity in both engineering and operations due to the US holiday. We also named Markus Glaser and Mark Hershberger
as the new contractors maintaining the MediaWiki “tarball” for release to other system administrators and organizations.
With the migration of pywikipediabot from Subversion to Git
, we were able to switch svn.wikimedia.org to read-only mode, thus completing this migration. We plan to keep the Subversion service around indefinitely for archival purposes, and can still migrate any dormant project that hasn’t already been migrated on request.
In July, we continued to expand our multimedia team
: Mark Holmquist joined as front-end software developer, working with product manager Fabrice Florin and engineering director Rob Lanphier, as well as contractors Brian Wolff and Jan Gerber. We prepared a first multimedia plan for the coming year
and discussed our goals with community members in two separate events: a multimedia roundtable
and an IRC chat
. Based on community feedback, we identified five main areas of activity 2013-2014: improving the viewing experience and upload pipeline in the first half of the year, then focusing on file curation, discovery and placement in articles for the second half of the year. Our overall goals for this year are to increase both the number of contributions and files used in Wikipedia articles. For now, we have started work on a new media viewer
to display images in larger size when you click on a thumbnail, as well as display file information and a full-screen viewing option, right on the same page. We plan to have a first version of that feature next month, and will be testing it as part of a beta experiment
on a few pilot sites. We will also be hosting more community planning discussions, such as this multimedia roundtable at Wikimania 2013
. To participate in these discussions and keep up with our work, we invite you to join this new multimedia mailing list
. Last but not least, we are also recruiting for two more positions for our team: a multimedia systems engineer
and a senior software engineer
. Please spread the word about this unique opportunity to create a richer multimedia experience for Wikipedia and MediaWiki sites!
Admin tools development
This activity was on hiatus in August.
Nik Everett and Chad Horohoe have continued writing an extension to implement ElasticSearch searching for MediaWiki, and we’ve finished most of the required features. Next comes getting it deployed, scaled, and fixing the inevitable bugs. We’re aiming to deploy to the test site beta.wmflabs.org before the end of the month. Peter Youngmeister and Asher Feldman will be handling the operations tasks for the new setup.
Engineers worked towards for an OAuth
deployment to the beta cluster
in early August, and aim to roll OAuth out to the test wikis (e.g., test2.wikipedia.org) after Wikimania.
HipHop work was mainly on hold in July
, with the exception of some minor work on virtual machines.
Security auditing and response
The team continued to respond to reported security issues, and addressing outstanding bugs.
This month QA made contributions to the VisualEditor, UniversalLanguageSelector and Mobile web projects, among others, finding and reporting issues in a timely manner. Our intern with the Outreach Program for Women
is working on more automated browser tests. We continue to engage our community on the QA mail list
and in live sessions, where we have several contributors (see Volunteer coordination and outreach
The Beta cluster continues to be a target for automated and manual testing. It also finally has a syslog receiver on deployment-bastion, thus solving bug 36748
(no syslog::server in beta). The logs can be accessed via either /home/wikipedia/syslog or /data/project/logs/syslog/ . This is thanks to Leslie Carr.
In July we added coverage for a number of features, including VisualEditor, UniversalLanguageSelector, and Mobile Search. We are making extensive use of beta labs as well as the test2wiki test environment. Our automated browser tests continue to identify important issues during feature development.
We reviewed our planning document with the Sue and Erik and the Engineering Directors. Reception was positive and we will be communicating next steps more widely in August. The Analytics team focused on short term deliverables, reliability and hiring in July. We identified two potential candidates for front-end/Python work. We have been performing multiple phone screens together with Recruiting, and the hiring pipelines are good.
- We kicked off a reliability project with Ops with the end goal of stabilizing Hadoop and the logging infrastructure. Teams have been in discussions on architecture and planning, and should have a path forward in the next 2 weeks. We identified a consultant who will perform a system audit to aid the project.
- We continue adding new metrics and alerts to monitor all the different parts of the webrequest dataflows into Kraken. We expect to keep making improvements in the coming months until we have a fully reliable data pipeline into Kraken.
- We started this month with designing a canary event monitoring system. A canary event is an artificial event that is injected at the start of the data workflow and which we will monitor to see it reaches its final destination; that way we can ensure that the dataflows are functioning.
- We are investigating what data format to use for sending the webrequest messages from Varnish to the Hadoop cluster. Formats that we are scrutinizing are JSON, Protobuf and AVRO, but we are also looking at compressions algorithms such as Snappy.
Analytics Visualization, Reporting & Applications
Wikimetrics: We successfully launched the initial version of Wikimetrics: see metrics.wmflabs.org. This version has support for cohort upload and two metrics: 1) bytes added and 2) namespace edits. We are working on adding support for time-series and aggregators. In the coming sprints we will focus on adding new metrics.
Wikipedia Zero: Dashboards have been moved off of Hadoop for the time being and are now being populated again. We have identified some issues with logrotation that are causing gaps in the graphs, and will look into these problems. Also, we have been working on technical handoff as Evan Rosen leaves the Foundation.
Limn: No development news.
Wikistats: No development news.
- Erik Zachte published data and longitudinal analyses of edit and revert trends for Wikimedia projects (read the announcement). We provided data and ad-hoc analysis for the presentation A State of Decline? The State of Wikimedia Communities as of July 2013 at the July 2013 Monthly Metrics Meeting.
- We published the analysis of a controlled experiment that we ran in June to test the Impact of notifications on new contributors and a pre-release A/B test of Visual Editor on the English Wikipedia. We performed an extensive audit of the quality of the data collected during and after the VE test, taking into account browser limitations and known bugs, and posted an update on the state of the analysis. We released via our open data repository the complete dataset of the sample of new registered users who participated in the split test to ensure the replicability of the analysis.
- We released real-time dashboards on edit activity, new account registrations and reverts for the 10 Wikipedias on which VE has been rolled out. (en • de • es • fr • he • it • nl • pl • ru • sv)
A PATCH_TO_REVIEW status was introduced in Bugzilla
which is automatically set (by the Gerrit Notification Bot) on bug reports when a commit message in Gerrit mentions a corresponding bug number. Andre prepared a patch for using the InlineHistory extension in Bugzilla
and a patch
to make Bugzilla’s guided bug entry form
for new users usable for Wikimedia Bugzilla. Andre also continued his weekly blogposts of Bugzilla tips
. Thanks to Daniel Zahn, Bugzilla administrators now regularly receive an email with a database dump of Bugzilla’s “audit log”
which lists the most recent taxonomy changes in Bugzilla (component or keyword additions, etc.). In Bugzilla’s taxonomy, the components in the “Parsoid” product
were reorganized as requested by its main developer, and the remaining open “OggHandler” tickets were closed as it has been superseded by TimedMediaHandler.
organized meetings with each Google Summer of Code
and Outreach Program for Women
team, one by one. Most projects were already at full speed, and for them, the meeting was primarily social and nice to have. A few really benefited from going through a checklist to highlight early problems easy to solve now. All GSoC and OPW projects, 21 in total, are now on track.
Like in June, Guillaume Paumier
was seconded to the VisualEditor deployment effort, working on communications, documentation and liaising with the French Wikipedia. Work on technical communications mostly focused on perennial activities like ongoing communications support to the engineering staff.
Volunteer coordination and outreach
On Community metrics
, Quim Gil
focused on the consolidation of korma.wmflabs.org
, the new dashboard for automated community metrics. We have made good progress on this alpha, including basic metrics from Git, Bugzilla and mailing lists being retrieved on a daily basis, and have filed bugs and enhancement requests on GitHub (mediawiki-dashboard
). We are deciding on the key metrics we need in order to make decisions, e.g. average time to resolve on Gerrit changesets or bug reports. We also planned and promoted a Browser Testing Automation workshop with Cucumber
together with the QA team, with 13 people participating online. You can watch the session here
(1h40). The experience was useful, as we agreed on MediaWiki-Vagrant
as the default environment for automated testing and highlighted the list of easy bugs
. Also, the Engineering Community team held its quarterly review