That would be interesting just to look at edits, but since our project was done in order to support identifying and mentoring new editors with accounts, it would not serve our purposes well. We kept in mind the change in the ability of anonymous IP editors to create pages, which is why I didn’t focus in the post on the change in the percent of vandalism. I agree that it’s probably why there is such a difference in the number of vandal accounts that registered.

Just to answer your second suggestion: no, we’re not really looking at editor retention comparatively over the years, since we’re already pretty clear about those from the Editor Trends Study. I do think that Philippe and James are tracking retention of editors selected for mentorship by WikiGuides though. The same goes for matching “unacceptable” edits and reverts by the community; we really just wanted to look at quality of newbie edits, so we stuck to that for last week’s experiment.

Thanks for your comments, they are all super insightful!

Nice findings. I also agree with the previous comments in that more info about the quality classification criteria would be great to put these results in context.

Suggestions:

1. It would be nice to increase a bit the size of the sample (although there is a trade-off with number of edits that can be assessed through manual inspection).

2. Do you plan to measure % of retention of editors according to outcome of the first edit? This might spot some light on the potential influence of first experience with the project to keep on contributing.

Quick question: is there a match between non-acceptable edits and reverts? That is, did you evaluated the edits independently or is this just what wikipedia editors decided about this first-time contributions?

Thanks for sharing!!

With regard to the study itself – I think this data becomes meaningful only when compared with the quality of edits of editors who are no longer new. Maybe they are also vandals?

Finally, I agree with comments above that a simple 1-5 does not suffice: how do you compare a significant edit to a small – but factually correct – correction, and that to a spelling edit? Also, have you used multiple coders to evaluate the edits?

I redrew the data as a bar chart. It’s easier to see the differences and similarities between the two data sets.

Wikipedia has a long tradition of encouraging “WikiGnomes” which mostly do these kinds of edits, so it’s a widely recognized kind of contribution in the community, even if we don’t always talk about it.