3:56 UTC – Domains have been added to all affected sites and they are now back online. Please contact help@pressable.com if you have any questions.

20:18 UTC – We continue to move files and databases into the restored sites. Once this process is complete, we are adding domains to the restored sites. We appreciate your continued patience and understanding.

17:37 UTC – Sites have been restored to the affected accounts. We are now in the process of adding the domains to the sites.

14:12 UTC – Progress continues to be made with the small amount of remaining sites to restore. Thank you for you understanding and we will continue to regularly update.

09:00 UTC – The engineering team has restored several sites. However, some of these sites may not have primary domains set or be in Production Mode. Once the primary domain is set and Production Mode is enabled, any issues you observe with domain redirects and missing JavaScript or CSS files should resolve themselves within approximately 10 minutes. Our apologies in advance for the inconvenience.

A very small number of sites may still be affected by this issue, however. We are working on restoring these sites and will post here once we have more information. Thank you for your continued patience and understanding. If you have questions or concerns, please contact the Help Desk: help@pressable.com.

08:12 UTC – The engineering team continues their work. We anticipate most affected sites to be functioning properly within the next hour.

07:12 UTC – Our engineering team continues to work on the restoration of the affected sites. We expect the majority of them to be online in the next 2 hours. We thank you for your continued patience.

06:12 UTC – Our engineering team is implementing a fix to restore all affected sites. We will have 80% of the impacted sites back online within 3 hours. We are exploring all methods of restore from our backups and will keep posting as information becomes available or every hour until this is resolved.

05:12 UTC – Our engineering team continues to work on restoring the affected sites. We appreciate your continued patience and understanding.

04:12 UTC – Our systems team is currently investigating an issue that caused some sites on our platform to become unavailable. This issue occurred as a result of routine maintenance. We have isolated the cause and our engineering teams are working to restore the affected sites as fast as possible.

Last Wednesday, April 18th, Pressable experienced an outage from an accidental fiber cut in our upstream provider’s connectivity. Over the past week we’ve worked with our upstream provider, as well as our engineering team, to ensure future outages don’t happen again.

We have evaluated and inspected the redundant fiber connectivity to our data center. During this process we confirmed the following important factors. First, the upstream provider had an undocumented and isolated area or fiber without redundancy. The outage occurred when the fiber in this area was accidentally cut.

Secondly, there is now 99.07% redundancy for the remaining connectivity. This provides confidence in our facility serving the uptime and availability you’ve come to expect from Pressable.

We are working as quickly as possible to get the final < 1% of our network fully redundant in the weeks ahead and will continue to update our valued customers as progress is made. In the meantime, if you have any questions feel free to reach out to me directly by emailing  Jay@Pressable.com .

UPDATE: 19:25 UTC phpMyAdmin is up and working at this time.

19:14 UTC We are aware of and investigating an issue with access to phpMyAdmin in my.pressable.com

12:40 UTC We are aware of and investigating a partial outage impacting some customer sites.

13:05 UTC Outage resolved. Investigation into cause of outage ongoing.

RESOLVED @ 8:33AM CST: Things have been stable for more than 8 hours. The root cause of the incident does appear to be a spike in traffic to our servers which caused a rather ungraceful failure of a couple load balancers.  We’ve identified some issues that will help prevent this scenario in the future and will be implementing them over the next few days.

UPDATE @ 12:18AM CST:  All sites continue to operate normally.  We will continue to monitor things throughout the evening.  Our initial findings show that this incident was most likely caused by an abnormal spike in traffic toward Pressable’s servers.

We are currently investigating an outage affecting a percentage of the sites on the Pressable platform.  Currently, everything is back up and operating as expected.  The partial outage lasted from around 11:00 Central to 11:30 Central time.  Once we have more information, we will update this post.

RESOLVED @ 3:22pm CST: The network issues have been resolved and everything is online at this time. We continue to investigate the root cause with our provider.

3:19pm CST: There is currently a problem with our hosting provider which is affecting all Pressable-hosted websites. We are actively troubleshooting the issue and will update this post with more details as they are available.

RESOLVED @ 3:54 AM Central: Our hosting provider isolated the cause of the problems and made changes that should prevent further service disruptions.  We are not anticipating another loss of connectivity and will continue to monitor the situation.

UPDATE @ 3:41 AM Central:  The network connectivity issues returned momentarily, but things have recovered. The root cause appears to be some network instability between Dallas and San Antonio, Texas.  Will provide more details once they are available.

UPDATE @ 3:34 AM Central:  The network issues have been resolved and everything is online at this time.  We continue to investigate the root cause with our provider.

3:14AM Central: There is currently a problem with our hosting provider which is affecting all Pressable-hosted websites. We are actively troubleshooting the issue and will update this post with more details as they are available.