Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Posts by Max Semenik

GeoData: a new age of geotagging on Wikipedia

On mobile devices, GeoData will allow you to view Wikipedia or Wikivoyage pages about places close to your location.

On mobile devices, GeoData will allow you to view Wikipedia or Wikivoyage pages about places close to your location.

Have you ever wondered if there are Wikipedia articles about things near you? Well, wonder no more! Today, we present the GeoData extension for MediaWiki, which now provides a structured way to store geo-coordinates for articles, as well as an API to make queries around this information.

What does it do?

Coordinates added to articles are now stored separately in the database, as opposed to being stored arbitrarily in wiki markup. This makes it easy to query the coordinates of a particular page or a list of pages around a given set of coordinates. The Solr search engine is used for spatial queries, making the searches extremely fast. All the functionality is also available via the API, allowing developers to create tools that use this data.

Where will it be used?

One of the first major uses of GeoData information will be in the experimental mode of the Wikipedia mobile site, which will allow beta users to see a list of nearby articles. We expect to release this feature into our experimental mode in the coming weeks. In the future, we also hope to build this functionality into our contribution tools: for instance, we’d like to be able to show users a map of articles needing images near their location, so that they can easily take and upload photos of subjects around them missing an illustration.

How does it work?

GeoData adds a new MediaWiki parser function called {{#coordinates}} that saves coordinates passed to it into the database. For example, if the Wikimedia Foundation office is located at 37° 47′ 13.09″ North, 122° 23′ 58.84″ West, it can be tagged like that: {{#coordinates:primary|37|47|13.09|N|122|23|58.84|W}} or with decimals: {{#coordinates:primary|37.78697|-122.39967}}. primary indicates here that these are the coordinates of the article’s primary subject, as opposed to other coordinates mentioned in the article.

How do we add it to articles?

GeoData requires some manual setup to start collecting data on a given wiki. The {{#coordinates}} tag needs to be inserted into the template used for geotagging (example change), and the job queue will process the pages that use it. This can be done on all sites where the GeoData extension is enabled: all Wikipedia and Wikivoyage sites, as well as special projects (like Commons). At least four projects are already actively using it: Wikipedia in English, German, Simple English and Chinese.

It’s also recommended to create a tracking category to which GeoData will add pages with invalid coordinates. The name of this maintenance category can be localized by editing the MediaWiki:Geodata-broken-tags-category system message. If you need help setting up GeoData on your wiki, join us on the #wikimedia-mobile channel on Freenode IRC, and we’ll be happy to help.

Max Semenik
Software Engineer, Mobile

Calling all mobile app developers: A better way to access Wikipedia

There are many third-party mobile apps based on Wikipedia, and some of them offer unique benefits. The Wikimedia Foundation encourages diversity and innovation using the free content of Wikipedia, as long as some legal guidelines are followed (namely, observe the CC-BY-SA license terms and don’t infringe our trademarks). We want app developers to help spread free knowledge, and now you can do that more easily and reliably.

Most Wikipedia app developers are screen-scraping the mobile site, or using the desktop site’s HTML and applying some home-made transformations to it. Our official apps used to screen-scrape, too, but now there’s a better alternative: over the last half year, the mobile team has developed a specialized API for mobile apps to use.

There are two problems with screen scraping: it’s brittle and slow. Here are the benefits of using the new API:

  • You don’t have to worry about unrelated changes to the mobile site breaking the app - Are you grabbing the content of a particular <div>? It may disappear with the next skin update. The API is guaranteed to remain backwards-compatible.
  • Structured dataSince the data is a bit more structured, you can easily able to do things like loading images only when their section is expanded.
  • Get just what you want, when you wantIt helps responsiveness a lot by loading only sections the user wants to see.
  • Improved speed, saved trafficDownload speed and bandwidth usage are improved because you are getting just the data you want, without extra formatting and navigation elements.

Our official Wikipedia apps switched over to using the new API, and we have received comments like this:

… the reason why I wrote this review is because of the latest update. It was supposed to make the app faster…. and it did! Seriously, the app loads pages MUCH faster now, and I am very pleased. We’re talking about going from internet explorer speed to any web browser that isn’t internet explorer speed. A+

Max Semenik
Software Developer (Mobile)
Wikimedia Foundation

FeaturedFeeds brings syndication feeds of featured Wikimedia content

Example of the English Wikipedia
featured articles feed generated by FeaturedFeeds

Yesterday, we deployed a new MediaWiki extension,  FeaturedFeeds, to all Wikimedia wikis. It creates syndication feeds (Atom or RSS) of Wikipedia’s featured content, such as featured articles or pictures of the day, giving the projects a new way to deliver content to readers and users.

For now, links to the feeds only appear in page metadata; in the future, we will add them to the sidebar on main pages, if communities wish so.

FeaturedFeeds integrates with the existing main page infrastructure: it uses data from templates to show content based on the current date.

Because user-generated content is involved, local wiki administrators need to make a few edits to MediaWiki pages to set up the extension. Instructions and a FAQ will guide you through the process. You can also use my edits to set up FeaturedFeeds on the English Wikipedia as an example.

If you have questions, you can ask for help on IRC, in #mediawiki and #wikimedia-mobile; we’ll be happy to help you set up the extension on your wiki.

Max Semenik
Mobile team developer