Mark Bergsma

  1. Wikimedia failover test—expected impact for editors

    Photo by Arild Vågen, CC BY-SA 3.0.

    We are launching a new data center in Texas, and to do so we will be placing Wikimedia sites into read-only mode for two short periods next week.... Read more

  2. Internal security incident identified and resolved at the Wikimedia Foundation

    Photo by Petar Milošević, freely licensed under CC by-SA 4.0.

    A security incident with the Wikimedia Foundation’s Mailman mailing list system was identified and addressed today. Photo by Petar Milošević, freely licensed under CC by-SA 4.0. On November 12, the Wikimedia Operations team identified a security incident on the Wikimedia Foundation’s Mailman mailing list system that resulted in the breach of four staff email accounts. We immediately inve... Read more

  3. Protecting users against POODLE by removing SSL 3.0 support

    "Pudel-drawing" by Gustav Mützel (Brehms Tierleben), public domain

    To protect our users against the recently disclosed POODLE security vulnerability, we are removing support for SSL 3.0 on all Wikimedia sites as of 15:00 UTC (8:00 am PDT) today.
    SSL 3.0 is an outdated implementation of the HTTPS web encryption protocol. […]... Read more

  4. Wikimedia Foundation selects CyrusOne in Dallas as new data center

    In October 2013, we launched a public request for proposals regarding a new data center location in the continental US for co-locating Wikimedia infrastructure. We’ve now concluded this process and selected the CyrusOne facility in Dallas/Carrollton. Wikimedia’s primary data center is in Ashburn, Virginia, and we’ve been preparing to end our remaining hosting presence in Tampa, Florida (whic... Read more

  5. Thumbnail issues being resolved

    Last Monday, our Solaris server that contains all image thumbnails developed problems. It ran out of memory, became too slow and eventually even started to crash. (For the technically inclined: we think the kernel is leaking some file system structure in kernel memory.) This caused missing thumbnails across Wikimedia projects. We addressed these problems in the following ways: We decreased the loa... Read more

  6. Wikimedia projects down due to power problem in primary data center

    Starting at 0:10 UTC on July 5th, the Wikimedia Foundation suffered from intermittent, partial power failures in the internal power network of one of its main data centers in Tampa, Florida. Due to the temporary unavailability of several critical systems and the large impact on the available systems capacity, all Wikimedia projects went down. The power situation stabilized at 1:12 UTC, and systems... Read more

  7. Global Outage (cooling failure and DNS)

    Due to an overheating problem in our European data center many of our servers turned off to protect themselves. As this impacted all Wikipedia and other projects access from European users, we were forced to move all user traffic to our Florida cluster, for which we have a standard quick failover procedure in place, that changes our DNS entries. However, shortly after we did this failover switch, ... Read more