Introducing Montage, the web platform used to help judge the world's largest photo competition

Imagine, for a moment, what it takes to organize a competition with a quarter of a million contestants. To make things more interesting, winners are selected from 40 different campaigns around the world, each with freedom to choose their own tools and design their own process . To top it off, do it year after year for the better part of a decade. Each year welcomes new people and new improvements to the processes.
If this sounds hard, then consider thanking your local Wiki Loves Monuments (WLM) volunteers. Thanks to the Commons community, 2016 just saw another successful iteration of the world’s largest photography competition. Every year, a distributed network of volunteers come together to organize Wiki Loves Monuments, the world’s largest photography competition. In 2016, these volunteers made the contest happen for the seventh year in 42 national competitions resulting in more than a quarter of a million photographs.
How do you build tools for this unconventional process while still keeping it flexible and open for organizers to decide how to run their competitions? This year, the international team of Wiki Loves Monuments (WLM) organizers set out to address this chicken-and-egg problem, with a little help from some volunteer work from Hatnote.
Today, we’re happy to present Montage, the web platform used to judge Wiki Loves Monuments 2016.

Overview

We started by looking at all the independent processes and tools used over the years. We wanted to make it easier than ever to organize a photography contest on Wikimedia Commons, and including communities who could not participate regularly. Before long, a new design emerged, one which would enable a standard recommended process, streamlined for future participants.
For those unfamiliar with the Wiki Loves Monuments competition, here is a handy executive summary:

1. Countries have one or more official lists of national or regional heritage sites. These lists make it onto Wikipedia with a little help from officials, volunteers, and bots.
2. Local contest coordinators self-organize on their language-specific wiki, then contact the international team to indicate their country’s participation.
3. These coordinators advertise the contest on their wiki, social media, and elsewhere.
4. Contest submissions are accepted for one month, around September, through a special Commons upload link, like this one, which places entries under a particular category, like this one.
5. When submission period ends, the 100 to 100,000 entries are judged by a jury selected by the local coordinators, consisting of experts in photography, history, anthropology, and the local wiki community.
6. After several rounds of judging on a variety of criteria, at least 10 winners are chosen from each local contest.
7. The top 10 winners from each locale, hundreds in total, advance to the international stage, where a specially selected jury judges them in several rounds.
8. The winners of the international stage are presented in a report, like this one from 2016’s contest, to much joy, fanfare, and celebration.

Submissions and prizes may vary from year to year, but the contest has always achieved its goals:

• Collecting invaluable, free-to-use heritage documentation on Wikipedia and elsewhere.
• Familiarizing a new set of creators with contributing to Wikimedia Commons.

It’s no wonder that Wiki Loves Monuments’ success has inspired many other “Wiki Loves” competitions, for all sorts of subjects, including nature, food, and folk culture.

Design

Considering the unique processes of Wiki Loves Monuments, Montage had to be designed specifically for photo contests hosted on Wikimedia Commons. As with all good software, Montage puts its users first, so let’s start by taking a look at them.

User roles

We took a hard look at the WLM process and determined that we had to build functionality for the following groups of users:

• Maintainers: The developers of Montage itself, and key organizers
• Organizers: The central WLM organization team
• Local coordinators: The teams formed around each local competition
• Jurors: A judge chosen by organizers
• The general public: In all likelihood, you, dear reader

Except for the public parts of the site, all users log in with their Wikimedia credentials.
Note that a person can take on more than one role, especially across campaigns. For instance, LilyOfTheWest coordinated Wiki Loves Monuments Iran, while also helping organize the international competition, and served as juror on a number of test rounds.
Each user role’s behavior in a nutshell:

• Maintainers add international organizers
• International organizers can create campaigns and add local coordinators
• Local coordinators can create rounds and add jurors for those rounds
• Jurors can view and vote on images in rounds they’ve been added to
• The general public can see our beautiful login page

So far, Montage has seen logins from over 200 users from all roles. Let’s take a peek into what their overall workflow looks like.

Workflow

The smallest WLM2016 campaign on Montage ran with 760 entries, two rounds, and five jurors. Compare that to the largest campaigns, which had tens of thousands of entries, up to six rounds, and almost 50 jurors. We’ve worked hard to design Montage for contests of any size.
Once a campaign has been created in Montage, organizers import any number of entries into a first round of voting, then run successive rounds, one at a time, to narrow down the field to just the set of winning images.
Montage supports three voting styles, providing flexibility to organizers without too much complication for jurors:

• A simple yes-no
• A five-star rating
• A first-to-last ordered ranking

A juror logging in, assigning ratings, editing ratings, and later ranking entries would see something like:

———

———

———

When a round’s voting is complete, organizers select the rating threshold to determine the set of entries advancing to the next round. Entries below the threshold are eliminated. Examples of thresholds would be:

• Yes-no: At least 2 “yes” votes
• Five-star: Average score of 3.5 stars or higher
• Ranking: Top 10

Because organizers select the threshold value from a fixed set, as seen below, Montage sidesteps complicated tiebreaking logic.

To lighten the load on individual jurors, while still ensuring each image gets a fair appraisal, Montage provides a quorum setting for yes-no and rating rounds. For example, in a rating round with quorum set to 3, each image must be rated by at least 3 randomly-selected jurors. This spreads the load for those early, large rounds.
For more in-depth detail about the research and design behind Montage’s voting system, see this post.

Technical details

Montage also packs some notable technological advantages.
First off, Montage is hosted on Wikimedia’s community server, Tool Labs. This supported environment makes collaboration easier, while also providing critical features like automatic database backups, direct access to database replicas for faster imports, and HTTPS. The price is right, too!
As mentioned earlier, Montage also uses Wikimedia’s OAuth for authentication, meaning that user accounts are autocreated on first login, as long as they’re registered on Wikipedia, Commons, or some other Wikimedia wiki. This means users don’t need to remember another password, and the Montage team doesn’t need to worry about storing sensitive user information. Votes, permissions, and other information is still treated confidentially, of course.
Montage itself is built on Python, Werkzeug, and Clastic, giving it an added measure of maintainability. It has extensive logging throughout, using Lithoxyl. On the frontend, Angular.js provides a responsive user experience with a familiar feel.
The code is free and open to contributions and issue reports on GitHub, of course.

New frontiers

Without a doubt, Montage was much more of a success than we had anticipated. In the span of just a couple months, Montage has raised the bar for software support, while lowering the barriers for future coordination.
Right now we are putting together our feature list for the next iteration of Montage. The priorities are far from final, but some directions we are investigating include:

• Adding more public-facing features to the site
• Improving internationalization and localization
• Enhancing communication between organizers and jurors
• Supporting lower bandwidth connections
• Further customization of photo viewing, including light/dark themes and zooming
• And as always, increasing performance!

The list is long, and we’re excited to tackle it for the 2017 Wiki Loves contests! We hope you’ll join us. If you’re an engineer of any sort, get in touch!

Thanks

Many people contributed to Montage directly and indirectly. To name a few:

• LilyOftheWest – Product and project manager
• Paweł Marynowski – UI designer and engineer
• Mahmoud Hashemi – Architecture and backend
• Stephen LaPorte – Backend and much more
• Jean-Frédéric – Design review
• Lodewijk – Leadership and review
• Yuvi Panda – Operations and support
• Ori Livneh – Operations and support
• Kenneth Samonte – Logo and illustration

And a huge thank you to all of the Montage users and Wiki Loves Monuments organizers and jurors. We couldn’t have done it without you!
We hope this peek behind the scenes of the contest has been enlightening. For more details and updates, feel free to watch the repo on GitHub, follow Hatnote on Twitter, and of course, watch/update/discuss via our Commons page. We look forward to hearing from you, and seeing your 2017 contest submissions!
Mahmoud Hashemi

All images and screenshots are courtesy of the Montage team. This post’s introduction has been rewritten after publication.