Global Outage (cooling failure and DNS)
Due to an overheating problem in our European data center many of our servers turned off to protect themselves. As this impacted all Wikipedia and other projects access from European users, we were forced to move all user traffic to our Florida cluster, for which we have a standard quick failover procedure in place, that changes our DNS entries.
However, shortly after we did this failover switch, it turned out that this failover mechanism was now broken, causing the DNS resolution of Wikimedia sites to stop working globally. This problem was quickly resolved, but unfortunately it may take up to an hour before access is restored for everyone, due to caching effects.
We apologize for the inconvenience this has caused.
Update: Unfortunately, for many, this outage seems to have lasted longer than an hour. It appears that many ISPs’ DNS resolvers do not honor the so-called Negative Cache TTL that we send (1 hour), and instead use a longer value. We have circumvented this problem by renaming the affected DNS record to something else.
Update 21:32 UTC: Our SSL gateway, secure.wikimedia.org, was disabled due to overload issues, but is now back up.
And so we remember Thursday, March 25, 2010 as the day every English speaking student failed their research papers.
Nice to see you getting back online!
How am I to study for my test without wikipedia!??!?
Who’s data center is that? Is it run by Wikimedia?
Damn, I have a paper due in an hour and a half…
Shit happens :-))
Nice to see you stil alive
All you slackers waiting until the last minute. If you fail to plan you should plan to fail
Given Wikipedia is free is it not our right to criticize when something fails.
The site is sorely missed and we stand that you solve this issue speedily.
Well done for everything you do to assist humanity.
For you needing to do research, just go to: archive.org and copy and paste the wiki url.
@1: not only english speaking ones…
Mirror:
http://encyclopedia.thefreedictionary.com/
Duh…use Google cache already…it has the wikipedia pages cached [w/o images of course, but something is better than nothing]!
EvoSwitch, I presume? Was it the innovative cold corridor solution that gave or what?
Bah, now what to do at work?
The following entries can be added to one’s local “hosts” file:
208.80.152.2 en.wikipedia.org
208.80.152.3 upload.wikimedia.org
I could load the site via proxy while it was down, but its working ok now.
Don’t use wiki for papers.
At 1824, I could not raise the server for the RSS feed.
@Mazen Abdel-Latif
Use a proxy if it still isn’t working for you, it worked for me on every proxy site I tried.
RSS problem resolved 1850 thanks