Alastair: We're currently experiencing a power outage in one of our datacenters, we're working as fast as possible to get the issue resolved and will update the status at status.rackspacecloud.com as soon as we have updates. I apologize for any inconvenience and thank you for your patience.
At least she didn't say "any inconvenience we may have caused you." Today is a rainy day in the cloud. :-'(
As of 12:35AM CST Rackspace Cloud engineers are seeing intermittent connectivity to our WC2 cluster in our Dallas - Fort Worth (DFW) and data center. We are working to resolve the issue as quickly as possible and will update the status post accordingly.
If you have any questions or concerns please contact our support via live chat or at 1-877-934-0407 international +1.210.581.040.
UPDATE: As of 1:15am CST, Rackspace Cloud engineers are still working to address the current connectivity issues. We are making significant progress and we will post another update here shortly.
UPDATE: As of 1:30am CST, service has been restored to our WC2 cluster. We are going to continue to monitor the situation closely. Additional updates to follow.
It's as if Rackspace has lost its key value proposition. The first time this happened, many of us thought "oh, it's a one time thing." But now that so many of us pay more money for something that goes down every few months, it's starting to get annoying.
In some ways, I actually feel sorry for the team at Rackspace. This hurts them big time.
Yes, but when 100% uptime and fanatical support are key to your branding, even a little downtime undermines the expectations of current and prospective customers.
They set a high bar for themselves. By saying that they have the lowest downtime in the industry, more eyes are looking at them to see that they do as they say. That's what makes this such a nasty situation.
It's certainly disheartening. These have all been power-related so far. Given the amount of money they should have spent on power systems, there's no reason this should happen. I've been to Savvis' facilities and they know exactly when a battery in their UPSes start to fail and how badly. Gensets get load tested regularly. Everything is fully redundant with instant failover. Yes, things can still go wrong, but they shouldn't be going wrong this often.
( I am not blaming slicehost but does "power outage" make any sense? Aren't there power backups? (and presumably an alert system for when primary power goes down?)
If anyone has worked in/runs a hosting service, please enlighten me).
no a power outage never makes sense. all systems should be on online battery backup with gas powered backup generators for outages longer than 5 minutes. These systems should be tested periodically to make sure that they are all in good working order.
I think I'd rather not have 100% uptime and have periodically tested systems, to increase reliability as long as I know when my server MAY go down.
But yeh; someone is probably getting a tongue lashing. Human error is so easy; even for experienced engineers and especially with live electrical equipment... (been there, done that, got the scars)
Reminds me of Liquid Web who copied rackspace calling it Heroic Support. If you need to be a Fanatic or hero you're doing it wrong. I think I'll start a web host and call it "competent" support.
Yeah, being in the managed hosting industry, I can say that having constant power "outages" of this magnitude is pretty unacceptable for a company of their size. Especially since its part of their marketing scheme!
Actually Posterous seemed to be up but they were not processing emails (perhaps that bit is the bit hosted in the cloud ?). I had to post via the web...
Ironic that we decided not to host our own blog so that we would have a working communications channel if our main site got taken out....
After the article on Facebook game scams, he was seeming to move past his previous attitude. Now he just looks like a raving mad man. It's not Rackspace's fault if you left a single point of failure on your site by hosting in one datacenter. If you want better reliability, then pony up the cash. You're making enough from just one of those ads to do it.
my slice power source is hybrid, so once the electricity went down, it switched to the solar module, unfortunately, it happened at night time, so this did not helped much
--------------------------------------
irc.freenode.com #slicehost - lets celebrate our down-time.
Alastair: We're currently experiencing a power outage in one of our datacenters, we're working as fast as possible to get the issue resolved and will update the status at status.rackspacecloud.com as soon as we have updates. I apologize for any inconvenience and thank you for your patience.
At least she didn't say "any inconvenience we may have caused you." Today is a rainy day in the cloud. :-'(
As of 12:35AM CST Rackspace Cloud engineers are seeing intermittent connectivity to our WC2 cluster in our Dallas - Fort Worth (DFW) and data center. We are working to resolve the issue as quickly as possible and will update the status post accordingly.
If you have any questions or concerns please contact our support via live chat or at 1-877-934-0407 international +1.210.581.040.
UPDATE: As of 1:15am CST, Rackspace Cloud engineers are still working to address the current connectivity issues. We are making significant progress and we will post another update here shortly.
UPDATE: As of 1:30am CST, service has been restored to our WC2 cluster. We are going to continue to monitor the situation closely. Additional updates to follow.