During the month of April, all Wikimedia wikis will be placed into read-only mode for two short periods—an action that will allow the Wikimedia Foundation’s Technology department to test services in a new secondary data center in Texas (referred to as “codfw”).
The new data center is a replica of our cluster in Virginia. The main purpose of this data center is to improve the reliability and failover capabilities of Wikipedia and all of our sites for users around the world. It maintains a full, up-to-date copy of the databases for Wikipedia and other projects, plus many other services. In case of any type of disaster at the main data center in Virginia, the Operations team expects to be able to transfer all traffic to the secondary data center in Texas within minutes.
Upcoming test for the new data center
Major pieces of our infrastructure have been successfully deployed or tested there with actual live traffic, but until now, the heart of our sites has been missing: MediaWiki itself. This is changing. Working together with teams in Technology and several outside of it, the Technology department is now ready to perform a failover test, during which we will transfer all application server traffic and tightly coupled service dependencies to the new data center for a minimum of 48 hours. At the end of the test, we will transfer it all back again.
This failover process is scheduled to happen during the week of 18 April, with the actual switchovers beginning on Tuesday, 19 April at 14:00 UTC and Thursday, 21 April at 14:00 UTC. Any changes to this schedule will be noted on our Wikitech calendar.
Effect of this test on editors and other contributors to our sites
Ideally we’d make this switchover without impact to our users, but limitations in MediaWiki prevent that at this time. At the start and end of this test, we will have to place all wikis in read-only mode for a short time. We expect this step to take approximately 15 to 30 minutes each time.
During the week of 18 April, we will be halting all non-essential code deployments. This means that the regular MediaWiki deployment process will be stopped, and no other non-critical deployments will be done that week.
The process for this is quite involved today, but this switchover test will give us information that we can use to make the process simpler, faster, and more secure in the future. We hope to not only greatly reduce the disruption for our users and the time needed to make the switch, but also to reduce the amount of manual effort necessary. We appreciate your patience while we improve this essential infrastructure that helps us to keep useful information from the projects available on the Internet, free of charge, in perpetuity.
Mark Bergsma, Director of Technical Operations and Lead Operations Architect