We are currently experiencing an outage in our Chicago Datacenter. All sites hosted in this data center are currently unresponsive and not loading.
We are addressing this issue now and will have things back up and operational shortly.
UPDATE 5:30PM CST: We’re still experiencing issues with the network backbone at our Chicago DC. We’re working with our provider to determine what the cause is and how we can mitigate this traffic to restore services.
UPDATE 6:05PM CST: We’re still working with our provider to determine the cause of the increased traffic along our backbone. We apologize for the delay in getting things back up and operational.
UPDATE 6:30PM CST: We’re still working with our provider to determine mitigating steps we can take for our internal network. We’ll provide an update in 30 minutes.
UPDATE 7:00PM CST: We’re currently engaging more senior members of our providers team to help troubleshoot the issues. We’ll provide an update in 30 minutes.
UPDATE 7:35PM CST: We’re still working with our provider to diagnose the current issues. We’ll provide an update in 30 minutes.
UPDATE 8:30PM CST: Our team is still working to mitigate the effects of traffic on our backbone. We’ve started to push out configuration changes which we expect to help reduce the impact, but it will be a bit longer before we know the impact of these changes. We’ll provide an update in 30 minutes.
UPDATE 10:05PM CST: Our team is still working to restore services. We’ll provide an update in 30 minutes.
UPDATE 10:40PM CST: Our team is beginning some work to bring servers back online and then will continue to evaluate issues. We do not expect services to return to normal at this time and will provide another update in 30 minutes.
UPDATE Jan, 10th, 2015 @ 12:30AM CST: Our team is still working on bringing servers back online and undergoing internal troubleshooting processes. We will continue to update the status blog as often as possible.
UPDATE Jan 10th, 2015 @ 1:25AM CST: We’re currently experiencing delays in the restoration of services as Rackspace is having provider line issues. We’ll continue our efforts as best as possible while Rackspace works to improve their provider issue. (https://status.rackspace.com/)
UPDATE Jan 10th, 2015 @ 2:45AM CST: The team is still working to bring services back online following issues from yesterday. We will continue to update the status blog as we make progress.
UPDATE Jan 10th, 2015 @ 6:05AM CST: Our team is still working to bring services back online. Some sites may have begun responding over the past hour, however, capacity is still not back to 100%. We’ll provide more updates as we progress.
UPDATE Jan 10th, 2015 @ 7:30AM CST: The team is still bringing services back online from the outage. We will provide more updates as we have them.
UPDATE Jan 10th, 2015 @ 9:00AM CST: We’re still working on bringing services back online related to this outage. We’ll provide more information as progress is made.
UPDATE Jan 10th, 2015 @ 10:00AM CST: Our team is still working to resolve issues related to connectivity and site availability. We’ll provide an update when we have more information.
UPDATE Jan 10th, 2015 @ 11:45AM CST: The team is making some configuration changes while we work to bring additional capacity online. We’ll provide an update when there is more progress.
UPDATE Jan 10th, 2015 @ 1:20PM CST: The team is making progress and we hope to have all systems back up and running soon. Some sites may have begun responding over the past hour, however, capacity is still not back to 100%. We’ll provide more updates as we progress.
UPDATE Jan 10th, 2015 @ 3:30PM CST: The team has made updates to the system that will help get things running and stable. Capacity is still not back to 100%. We’ll provide more updates as we progress.
UPDATE Jan 10th, 2015 @ 6:20PM CST: Sites are beginning to be served now but you may encounter the intermittent 502 error while things continue to settle. We’ll provide more updates as we progress.
UPDATE Jan 10th, 2015 @ 9:20PM CST: Customers with sites on our “Thor” cluster should see their sites being served properly now. We’re finalizing some work on our “Bode” and “Galaxy01” clusters and expect those to be functional shortly. We’ll provide more updates as we progress.
UPDATE Jan 11th, 2015 @ 9:15AM CST: We wanted to get another note out to let everyone know that we’ve seen stability restored (as of last night) and are currently seeing all systems online. We sincerely regret the experience provided over the last several days, but do appreciate your continued patience. Tomorrow we’ll be providing a more detailed analysis of the issues and our steps to correct these issues moving forward.