A bit of background
In April of 2014 I found myself digging deep into analytics in search of possible improvements and insight into what we do as a chapter. What brought me there? One of our most renowned programs, Wikimedians in Residence. A Wikimedian in Residence (WIR) is a person who, as a Wikimedia contributor, accepts a placement within an institution to facilitate open knowledge in a close working relationship between the Wikimedia movement and the institution. They work to facilitate content improvements on Wikimedia projects, but more importantly serve as ambassadors for open knowledge within the host organization.
Wikimedia UK has been involved with WiR in the UK with varying degrees of support and supervision. Since the creation of the chapter, we always felt that the program was worth running, seeing it as one of the key ways we can engage with external organizations. However, I never knew for sure, if that was just a feeling. Toward the end of 2013 we decided to explore these notions.
Why and how to evaluate
As I focused on my questions about program impact, I embarked on a review process of the program, which eventually included: a questionnaire for all the key parties, online surveys, meetings, group discussions, the analysis of existing materials (e.g. residents’ reports) and creation of a review document.
In January of this year I planned to survey the Residents and host institutions about their views on the program. Since I wasn’t sure what to ask, I reached out to the Program Evaluation and Design team for help.
Their stringent approach was worth it. We boiled down the issues around what I actually wanted to find out from the survey. Doing that before creating the questions was a revelation to me. The questionnaire went much deeper than I had originally anticipated. This meant that when we worked on creating the survey questions, every point was there for a specific reason and in a sensible order. With their help, I developed three surveys: for residents, residency hosts and another for community member input.
I was impressed with the amount of feedback that was shared. The Residents were clearly committed to the project and keen on telling me what could make the program more successful. At the same time I ran interviews with the host institutions. By that stage I was deeply entrenched in the review process. Discovering more about the program increased my appetite for a deeper analysis. This culminated in an April brainstorming meeting aimed at completing an analysis of the strengths, weaknesses, opportunities and threats (SWOT analysis) of our Wikipedian in Residence program.
With the data collection completed, I then examined all the reports and case studies produced by the residents and summarized them in terms of the impact made to Wikimedia projects. (Click here to read Overview of the residencies.)