We will be closed on Thursday, November 22nd in observance of the Thanksgiving holiday. If you are experiencing an outage or downtime please email our help desk (help@zippykid.com) with the subject line: Thanksgiving Emergency. We will respond to downtimes and will be monitoring our servers throughout the day. Non-critical issues will be answered on Friday, November 23rd.
We’ve noticed some major hardware issues on cluster 12, which is affecting around 100 websites. We’re working to fix this as fast as we can. We experienced this issue while doing prep work for the migration of this cluster to our TNG system. The transfer of this system to TNG is 95%, so we’ll be working to restore the sites onto the TNG platform by 7am CDT. Sorry for the issues, once this cluster is back, it’ll be much faster than before.
We’re currently experiencing an issue with the database cluster that is impacting site availability. We’re working with Rackspace to isolate and address the issue.
We’ll provide an update to the outage as we learn more. If you have any questions, please email help@zippykid.com.
A load balancer on server 17 went out early this morning as a part of an outage in one of Rackspace’s cloud load balancer clusters.
UPDATE 12:10PM: The load balancer for server 17 is still down at the moment and we are working with rackspace to have service restored as quicky as possible UPDATE 1:31PM: We are in the process of moving all sites on this server to our next generation architecture. Once this move is completed, sites will begin coming back up.We’re experiencing issues on our Classic environment. We’re investigating the issue right now.
We’re currently investigating an issue with the database cluster that’s causing connection issues. We’re working to restore stability to the system and we’ll update this post when we know more.
UPDATE 8:58PM CST: The database cluster is stable now. If you’re continuing to experience issues, let us know.
We’ve fixed the database glitch. All systems are back to normal.
We’re aware of some anomalies with our database cluster. We’re looking into it and will update you shortly.
Update as of September 17, 2012 12:22 AM : The issue has been resolved, all systems are back up and running.
Based on reports from various media organizations, there appears to be an issue with GoDaddy and at a minimum their DNS service (source, source, source)
What this means for you:
Hopefully GoDaddy is able to resolve the issue and return service to normal for the users that are being impacted. As always, if you have any questions, please feel free to email help@zippykid.com and let us know.
We’re currently experiencing an issue that’s caused our database connection to go offline. This is impacting the connection for all sites to the databases. We’re actively working to correct this, however, we expect this issue to last for about 20 minutes.
We do apologize for the inconvience and if you have any questions, please don’t hesitate to email help@zippykid.com.
UPDATE 5:35PM CST: As we continue to evaluate the situation we’re expecting our earlier estimates to increase some. We’re hoping to have things up and running within the next 45 minutes. We’ll let you know if this changes, and our sincere apologies for this.
UPDATE 6:20PM CST: We’re still working to restore the connections to database servers. There are some delays in getting things recreated, but we’re hoping to start restoring connectivity in the next 45 minutes (I know… we said this last time…). We’ll update you when we have some more information.
UPDATE 7:00PM CST: We’re starting see database connections for the sites come back online. We’re expecting to see service return to normal shortly. If you’re still having issues with a site, please let us know and we’ll take a look. Our apologies for the issues, and if you have any questions, don’t hesitate to ask.
