We’re now running all Wikimedia sites on our internal deployment branch of MediaWiki, so we’ve got a consistent record of what software we’re running and can more easily test it offline. There were a couple brief glitches during the switchover, but everything seems to be running smoothly now.

Additionally, we’ve started the process of moving thumbnail images from our primary upload file server to another machine to free up space and reduce the repetitive slowness problems we’ve been seeing lately as the ZFS data pool got near full. Even with an updated OS kernel we’ve been seeing it come back. :(

The transition of thumbs should be smooth and fairly non-disruptive in the background.

Update 2009-07-17 00:22 UTC: With the file server getting hyperslugging again, we’ve temporarily re-disabled uploads to speed up the transfers. Should be back online shortly.

Update 03:31 UTC: Uploads re-enabled after disk space rearranged.

Brion Vibber, Lead Software Architect