Over the next couple of years, the Wikimedia Foundation will be building capacity among program leaders around evaluation and program design. A better understanding of how to increase impact through better planning, execution and evaluation of programs and activities will help us to move a step closer to achieving our mission of offering a free, high quality encyclopedia to our readers around the world.

With this in mind, we are pleased to announce the first Program Evaluation and Design Workshop, on 22-23 June 2013 in Budapest, Hungary.

We have only 20 slots available for this workshop and the application deadline ends on May 17th. This two-day event will be followed by a pre-conference workshop at Wikimania 2013. Ideally, applicants would commit to attending both events.

The first Program Evaluation & Design workshop will be held in the shadows of the Buda Castle, Budapest, Hungary

Our long-term goals for the workshop are:

• Participants will gain a basic shared understanding of program evaluation
• Participants will work collaboratively to map and prioritize measurable outcomes, beginning with a focus on the most common programs and activities
• Participants will gain increased fluency in common language of evaluation (i.e. goals versus objectives, inputs and outputs versus outcomes and impact)
• Participants will learn and practice how to extract and report data using the UserMetrics API
• Participants will commit to working as a community of evaluation leaders who will implement evaluation strategies in their programs and activities and report back at the pre-conference workshop at Wikimania 2013
• …and participants will have a lot of fun and enjoy networking with other program leaders!

We will publish a detailed agenda for the event in Budapest soon on Meta-Wiki.

During the workshop in Budapest, we will only have a limited amount of time. Therefore, we will be focusing on the some of the more common programs and activities:

• Wikipedia editing workshops where participants learn how to or actively edit (i.e. edit-a-thon, wikiparty, hands-on Wikipedia workshop)
• Content donations through partnerships with galleries, libraries, archives and museums (GLAMs) and related organizations
• Wiki Takes/Expeditions where volunteers participate in day-long or weekend events to photograph site specific content
• Wiki Loves Monuments, which takes place in September
• Education program and classroom editing where volunteers support educators who have students editing Wikipedia in the classroom
• Writing competitions, which generally take place online in the form of contests, the WikiCup  and other challenges – often engaging experienced editors to improve content.

Contributors who play an active role in planning and executing programs and activities as described above in the Wikimedia community are highly encouraged to apply. Your experience and knowledge will make this workshop a success!

Hotels, flights and other transportation costs will be the responsibility of your chapter; the Wikimedia Foundation will provide the venue, handouts, breakfasts, light lunches, and a dinner for all participants on Saturday. If you’re not affiliated with a chapter and cannot afford to attend the event, please email me after you apply – we have a small amount of money set aside for those cases.

Remember, applications are open until May 17. You can apply via this Google Form.

Thanks for your interest, and I look forward to a great group of participants!

Sarah Stierch, Program Evaluation and Design Community Coordinator, Wikimedia Foundation

This post is available in 2 languages:

Português  • English

In English:

Professor Rafael Pezzi, a professor at the Universidade Federal do Rio Grande do Sul, read about a new program to support university faculty members in Brazil who were interested in incorporating Wikipedia editing into students’ assignments. Rafael was intrigued by the idea, and thought it might be a good idea for his physics course for engineering students.

“Wikipedia makes it explicit how knowledge is built: a never-ending dynamic process with conflicting opinions in a lively discussion,” he says. “Although this process is fundamental, it is ignored in the standard textbook-based class where knowledge is just given, considered as an absolute truth.”

Rafael was impressed with the level of support he received in the program, both from colleagues and from the Wikipedia community. Fabio Azevedo and Israel Rocha served as Wikipedia Ambassadors for his class. They introduced Wikipedia and motivated the students to participate in the assignment. They were also very supportive on-line, clarifying technicalities of the Wikipedia interface and introducing the participants to the Wikipedia community.

For Fabio, an editor since 2007, becoming a Wikipedia Ambassador and supporting Rafael’s class was a natural choice.

“I was motivated by using a modern teaching technique at my university, improving free content in my language, and helping Wikipedia to grow,” Fabio says. “I gave a talk on Wikipedia editing for his students in the first or second week of class. After, I replied to some emails, helping students with editing question and uploading images.”

“I must say that the help of the Ambassadors was key to making it happen,” Rafael says. “Without them it would not have happened, as I am not an experienced editor and created my Wikipedia account for this project.”

Rafael’s students really took to editing, creating or significantly expanding several important articles about physics related topics on the Portuguese Wikipedia, including articles such as Thermometer, Supercritical fluid and Interference of waves. In fact, five of Rafael’s students enjoyed the assignment so much that they volunteered to be Ambassadors for the current term.

“Basically traditional writing assignments are read once or twice and then sits in a drawer or folder archive in the professor computer,” Rafael says. “It is of limited value and does not really motivate the student apart from getting a good grade. With Wikipedia, the students have the real publishing experience and the feeling that what they write can help others understand what they are studying. Some get really motivated by this challenge.”

LiAnna Davis
Wikipedia Education Program Communications Manager

(more…)

At Rice University, students pursuing a minor in Poverty, Justice, and Human Capabilities (PJHC) must complete two core courses addressing poverty, justice, and human development. Since Spring 2010, Professor Diana Strassmann has given students in her courses the same final assignment: to create or expand Wikipedia articles about poverty and about the links between gender equality and economic development in various regions of the world. She has also trained the other professors who teach the minor’s core courses so that they can include the Wikipedia assignment.

Nadhika Ramachandran, a rising senior studying political science and international relations in addition to the PJHC minor, signed on for the first of the two core courses in Spring 2012 and learned that she would be editing Wikipedia. Nadhika was both excited and nervous.

“So many people turn to Wikipedia for information that you know your contribution can have a real impact in terms of how people view a certain subject,” she says. “Of course, that also made me a bit nervous because if I did not include certain information or an important viewpoint I would misinform people.”

For that term’s class, she significantly expanded the peacebuilding article, which introduced her to Wikipedia editing. Since Professor Strassmann has set this assignment up for both of the two core courses in the minor, Nadhika enrolled in the second class a year later. In that class, Nadhika created the Women in the Arab Spring article because she “felt it was an important issue that had no coverage” on English Wikipedia. The article explores women’s involvement in the political protests and demonstrations, including their role online.

“I love knowing that my work will educate people all over the world about an important but often-ignored topic,” Nadhika says. “The Arab world has a reputation of treating women as second-class citizens in areas like political participation, economic independence, personal freedoms, and general social status. When the Arab Spring first began, it seemed like a unique opportunity to boost the status of women in the Arab world. The protesters were pushing for democracy, increased political participation, respect for human rights, and better economic opportunities, all of which would improve the status of women. Additionally, women actually participated in the revolution as street protestors and in some countries, as leaders. However, as the new governments formed and Islamist parties won elections in most places people began to fear that they would actually curtail women’s rights. The impact of the Arab Spring remains to be seen.”

Nadhika’s class was supported by Wikipedia Ambassadors, who helped her and her classmates learn how to edit Wikipedia. The students received in-person support from university staff member Christine Cox as well as Virginia White and Joyce Chou, students who had taken the class in previous semesters. They could also seek assistance from long-time Wikipedia editors Mike Christie, Justin Knapp, and Pat Earley.

“The dedication of the other editors in the community reinforced the impact of our work and their support made me feel more comfortable when editing,” Nadhika said. After spending a few days responding to those editors’ constructive feedback, she submitted the new article to appear on the main page of Wikipedia as a Did You Know, and it appeared on March 25, with more than 1,500 views. Soon after creating the article, she was already able to achieve her goal of sharing the fruits of her academic studies with more people.

Thanks to Professor Strassmann’s advocacy for Wikipedia assignments, many students at Rice are editing Wikipedia during multiple terms. The students don’t necessarily edit in between assignments, but they’re returning to Wikipedia with a stronger editing background and familiarity with norms, so they have more time and energy to create even better content. Even though Nadhika has completed her class assignment, she plans to expand the section about the aftermath of the Arab Spring protests once her schedule clears at the end of the term.

Jami Mathewson
Wikipedia Education Program United States and Canada Associate

This post is available in 2 languages:

العربية  • English

In English

With more than 280 million native speakers, Arabic is one of the world’s most spoken languages, but the Arabic Wikipedia has lagged behind other language Wikipedias in terms of the amount of articles. The Arabic Wikipedia has only 205,000 articles — a tiny fraction in comparison to the English Wikipedia, which has 4.2 million articles. But the Arabic Wikipedia has been steadily growing over the last year, thanks in part to the efforts of college students in Egypt participating in the Wikipedia Education Program.

The Wikipedia Education Program kicked off in Egypt with a Cairo pilot at two universities, Ain Shams University and Cairo University. The chart at right shows the amount of content added to the Arabic Wikipedia by students participating in the program. In the first term of the pilot, students added about 1.85 million bytes of content to the Arabic Wikipedia — an incredible achievement celebrated at a conference in Cairo in July 2012. In the second term of the pilot, which wrapped up in February 2013, students contributed even more, with over 5.97 million bytes of content added to the Arabic Wikipedia. In addition, students who we’ve introduced to editing through the Wikipedia Education Program have contributed an additional 515,000 bytes, meaning the program has brought a total of more than 8.34 million bytes to the Arabic Wikipedia.

Volunteer program leaders organized a second celebration conference at Cairo University on February 27, 2013. Dr. Abeer Abd El-Hafez, a professor of Spanish from Cairo University, opened the conference and spoke about the spirit of the program and its importance in the lives of students and teachers in terms of skills development and new experiences. Faris El-Gwely, the education program consultant who runs the program in Egypt, shared results from the second term, and the best students and Ambassadors from the program received certificates recognizing their hard work. Students and professors also shared information about their experiences in the program. See more photos from the conference.

The second celebration conference was a catalyst for the program to grow. Two more universities in northern Egypt have joined the program, Damanhour University and Kafr El-Sheikh University, as has Saint Khadija High School for Girls in Cairo. The drive from these programs comes from past students and Wikipedia Ambassadors who want to volunteer their time to further the spread of the Wikipedia Education Program in Egypt:

• Walaa Abd El-Moneim, leader of the Faculty of Arts, Cairo University
• Doaa Saif El-Din, leader of the Faculty of Al-Alsun (Languages), Ain Shams University
• Helana Raafat and Mina Saber, leaders of the Faculty of Arts, Ain Shams University
• Samir El-Sharabaty, commander of the Faculty of Education, Damanhour University

Egypt is not the only Arab World country to see growth in the Wikipedia Education Program. In the term that’s just beginning, universities in Algeria, Jordan, and Saudi Arabia have joined the program. All told, more than 45 classes will be editing the Arabic Wikipedia as part of their coursework this term.

In Jordan, Dr. Nidal Yousef of Isra University is one of five university professors teaching Wikipedia classes this term through the program. The Jordanian Teachers Association is also leading a program where high school students in every governorate in Jordan will be editing Wikipedia as part of their schoolwork, assisted by volunteer Wikipedia Ambassadors. Dr. Abd El-Haq Fareh of Algeria is also incorporating Wikipedia editing into his free software class this term. And Dr. Mohammed Alghbban and Dr. Sami Bin Slimah of King Saud University are leading a Wikipedia translation program in Saudi Arabia in their school’s languages department.

We look forward to seeing the Arabic Wikipedia continue to grow, thanks to these dedicated faculty leaders and students.

Faris El-Gwely, Education Program Consultant, Arab World
LiAnna Davis, Wikipedia Education Program Communications Manager

(more…)

More than 40 people from 25 countries gathered together in person in Milan, Italy, last week to discuss Wikimedia projects’ use in education. Representatives from Wikimedia chapters, the Wikimedia Foundation, and universities worldwide discussed ways to further develop the relationships between educational institutions and Wikipedia and other Wikimedia projects.

Participants in the Education Program Leaders Workshop in Milan.

The Education Program Leaders Workshop was held in conjunction with the Wikimedia chapters conference in Milan, an annual opportunity for representatives from around the world to meet in person to discuss the future of the movement. The enthusiasm worldwide for the program bodes well for the future of Wikimedia projects like Wikipedia and education.

Notes from the workshop highlight the incredible depth and breadth of activities happening worldwide in the education sphere. Some programs, like in Serbia, Czech Republic, Ukraine, Brazil, and Egypt, have been in operation for several terms and have been achieving incredible results on their language Wikipedias. Others, including programs in Germany, Sweden, and the United Kingdom, have dedicated staff people working on furthering their goals. Programs in Mexico, Switzerland, and Saudi Arabia are small but effective thanks to the dedicated work of individual volunteer educators whose drive to use Wikipedia in their own classrooms has furthered their language Wikipedias. Still others are just getting started, and many are exploring opportunities to collaborate with governmental bodies who work on creating curriculum and education policy to include Wikipedia and other Wikimedia projects.

“Education” is a broad field, and participants represented programs working with everyone from school-aged children to seniors. Workshop participants discussed the different activities relevant to education programs, and talked about the best way of setting goals for programs as a whole. The Wikimedia Foundation remains committed to supporting education programs worldwide through such support resources as brochures, a MediaWiki extension, and online trainings. Workshop participants agreed that developing a better system to share experiences across countries — perhaps a searchable database of learnings — would help programs learn from each others’ mistakes and determine the best path forward for their own programs. With more than 30 programs in operation worldwide, the future is bright for Wikimedia projects and education.

LiAnna Davis, Wikipedia Education Program Communications Manager

Most Wikipedians I know – myself included – care deeply about our mission. We are highly passionate about our vision to provide free knowledge to every single person on the planet. And many of us invest an incredible amount of our free time into making this vision come true. Even though I have been editing Wikipedia since 2005, I’m still amazed when I look at the daily stream of Wikipedia’s recent changes. And I have a deep respect for the volunteers who invest their precious time and energy into never letting this stream of small and big improvements run dry.

For many years now, Wikipedians have not only worked on increasing and improving the amount of free content available on the web. We have also launched a wide variety of programmatic activities that intend to strengthen free knowledge by rising the public awareness of Wikipedia through exhibitions and presentations, by recruiting new editors as part of Wikipedia workshops, and by starting programs like “Wiki Loves Monuments”, the popular Wikimedia Commons photo competition.

We have not, however, been very strong at measuring the impact of those programmatic activities. “Measuring impact” in this case refers to quantifying the long-term effects those programmatic activities have on Wikipedia and its sister projects. In practice, this means, for example, analyzing how many of the people who attended a specific Wikipedia workshop actually turned into Wikipedians. How many of them embarked on editing articles as a result of the workshop and how many of them continued doing so on a regular basis?

Here’s where program evaluation comes into play.

• If you’re supporting programmatic work as a volunteer, you most likely want to know whether your activities are worth the effort. Whether you help running photo competitions or sign up as a speaker at workshops for Wikipedia beginners – you want to know whether you’re making a real difference with what you’re doing. Program evaluation will help you answer this question. By measuring the outcome of the programmatic activity you signed up for, you will know whether your photo competition was successful and whether the people who participated in your workshop really learned the skills they need to write great articles on Wikipedia. This knowledge will make your work as a volunteer more fulfilling.

• If you’re helping to improve existing programs, you’re most likely eager to find out which changes will make your program more effective and efficient. Imagine you could achieve the same result with fewer volunteer hours being spent. And what if you could double the number of people who actually start contributing to Wikipedia after your workshop, with making your workshop more engaging and fun? Improving an existing program requires that you measure its effectiveness. Here’s where integrating evaluation into your program design can make a difference.

• If you’re thinking about starting a new program, you will want to have some evidence that your new program is working. How else would you convince others to participate in your program? And how else would you convince a funder to provide you with a grant so you can eventually execute your program and grow it over time? Program evaluation will help you to make a strong case for your idea. And it will also prevent you from embarking on activities that have no impact.

• If you’re serving on the board of a Wikimedia chapter or a thematic organization, you might want to know which kind of programmatic activities produce the “biggest bang for the buck”. You might ask whether it makes more sense to start putting money and efforts into in-person workshops as compared to spending resources on creating an online training. How many hours of volunteer or staff time are you going to need in order to produce a specific result? Are the in-person workshops going to be more effective than the online training? And which of the two options will be more efficient? Also, which one is going to have the bigger long-term impact? In this sense, program evaluation can be a decision-making tool that will help you to determine which programmatic activities you want to embark on.

Finally, with the Funds Dissemination Committee (FDC) being in place since last year, there’s also another reason why program evaluation will be more important than ever: after the first round of funding for 2012/2013, the FDC requested more information about program impact, so it has a better foundation for making recommendations on what to fund in the future. This means that from now on, funding decisions will rely heavily on the ability of grantees to demonstrate what impact their programmatic activities have. That means that grantees will have to start thinking about program evaluation, in case they plan to apply for movement funds through the FDC process.

I’ve started a series of documents on Meta (“Program evaluation basics”) aimed at providing program leaders with an introduction to the main terms and concepts. Currently, three documents are available:

• Why evaluate in the first place? (covering most of what I mentioned above)
• Efficiency, effectiveness and impact (defining those three terms in the Wikimedia context)
• The program impact model (providing a basic model for how programs generate impact)

I invite you to take a look at the documents and to share your thoughts with me. I will also be available for an IRC office hour on Thursday, March 21, at 17:00 UTC. Let’s start talking about program evaluation…

Frank Schulenburg
Senior Director of Programs, Wikimedia Foundation