At approximately 7:24 PM Central on 4/11 (00:24 UTC on 4/12), we experienced a brief outage for some sites on our network. We isolated the issue and site availability was restored to normal at approximately 7:30 PM Central (00:30 UTC).
We will continue to monitor the situation but do not expect further issues. If you have any questions or concerns, please contact our help desk by submitting a ticket from your https://my.pressable.com control panel.
UPDATE: 19:25 UTC phpMyAdmin is up and working at this time.
19:14 UTC We are aware of and investigating an issue with access to phpMyAdmin in my.pressable.com
12:40 UTC We are aware of and investigating a partial outage impacting some customer sites.
13:05 UTC Outage resolved. Investigation into cause of outage ongoing.
RESOLVED @ 8:33AM CST: Things have been stable for more than 8 hours. The root cause of the incident does appear to be a spike in traffic to our servers which caused a rather ungraceful failure of a couple load balancers. We’ve identified some issues that will help prevent this scenario in the future and will be implementing them over the next few days.
UPDATE @ 12:18AM CST: All sites continue to operate normally. We will continue to monitor things throughout the evening. Our initial findings show that this incident was most likely caused by an abnormal spike in traffic toward Pressable’s servers.
We are currently investigating an outage affecting a percentage of the sites on the Pressable platform. Currently, everything is back up and operating as expected. The partial outage lasted from around 11:00 Central to 11:30 Central time. Once we have more information, we will update this post.
RESOLVED @ 3:22pm CST: The network issues have been resolved and everything is online at this time. We continue to investigate the root cause with our provider.
3:19pm CST: There is currently a problem with our hosting provider which is affecting all Pressable-hosted websites. We are actively troubleshooting the issue and will update this post with more details as they are available.
RESOLVED @ 3:54 AM Central: Our hosting provider isolated the cause of the problems and made changes that should prevent further service disruptions. We are not anticipating another loss of connectivity and will continue to monitor the situation.
UPDATE @ 3:41 AM Central: The network connectivity issues returned momentarily, but things have recovered. The root cause appears to be some network instability between Dallas and San Antonio, Texas. Will provide more details once they are available.
UPDATE @ 3:34 AM Central: The network issues have been resolved and everything is online at this time. We continue to investigate the root cause with our provider.
3:14AM Central: There is currently a problem with our hosting provider which is affecting all Pressable-hosted websites. We are actively troubleshooting the issue and will update this post with more details as they are available.
We just finished dealing with an issue in our Chicago datacenter that was causing several other clusters to experience instability. Our “Ursa” cluster was taking on an extreme amount of traffic that looks to be, in large part, a bot attack.
This happened as a result of the “Ursa” cluster having a set of tools not running appropriately that detects and mitigates issues like this.
We’ve cleared this up and all sites are now back up and running appropriately.
If you have any questions or concerns, please contact our help desk via your https://my.pressable.com control panel.