Category Archives: General Issues

Provisioning / Control Panel Issues

One of the machines behind our provisioning system has experienced a hardware failure, we are working on replacing the underlying hardware, and expect the machine to be back online in very soon.

In the mean time, functionality like adding, editing, removing a site, or collaborators of a site is not going to work. We apologize for any inconvenience.

Please note: This only affects our control panel, not your actual website.

Chicago Datacenter Issue – RESOLVED

Howdy,

We just finished dealing with an issue in our Chicago datacenter that was causing several other clusters to experience instability. Our “Ursa” cluster was taking on an extreme amount of traffic that looks to be, in large part, a bot attack.

This happened as a result of the “Ursa” cluster having a set of tools not running appropriately that detects and mitigates issues like this.

We’ve cleared this up and all sites are now back up and running appropriately.

If you have any questions or concerns, please contact our help desk via your https://my.pressable.com control panel.

Thank you!

Server Outages

Howdy,

We are currently experiencing an issue with sites being down on a small portion of our network. This issue is a result of some of our servers going offline unexpectedly. We are working with Rackspace on getting these machines back up right now and things will resume working normally shortly.

If you have any questions or concerns, please contact our help desk via the https://my.pressable.com control panel.

UPDATE December 4th, 2014 @ 8:40AM Central: A large majority of servers are back online and functioning properly. We have a couple more servers waiting to come back online but most sites are back up and functioning properly again. We will update once more as soon as all servers are back online and functional.

UPDATE December 4th, 2014 @ 9:00 AM Central: All servers and sites are now back online and functional. If for some reason you are still seeing a site offline, please let us know and we can figure out what is causing it.

Sporadic Functionality of Deploys and Clones

We are currently experiencing an issue that is causing clones and new site deploys to function sporadically.

Some customers are experiencing issues cloning sites and reporting unfinished clones. New site deploys are functioning for a large majority of our customers but there are still cases where new site deploys are not working.

We are attempting to resolve these issues as quickly as possible and will update this status blog as more information comes available.

If you have questions or concerns, please contact us by submitting a help desk request via your https://my.pressable.com control panel.

UPDATE Nov. 21, 2014 @ 10:41AM CST: Deploys across our systems are now working properly effective immediately. We are still working on fixing cloning across the board. We discovered that clones broke partially due to a fix we put in for POODLE recently. We will update again once this issue is resolved.

UPDATE Nov. 25, 2014 @ 9:20AM CST: Deploy functionality continues to work properly across our systems. The team is still working to resolve the incompatibilities with our cloning functionality and the increased security put in place post-POODLE.

UPDATE Dec. 1, 2014 @ 10:38AM CST: We are continuing to work on fixing the cloning issues from this past week. We have removed all clones that failed while this has been happening. We are asking for customers to refrain from attempting to clone until this issue has been patched fully. We will update again as soon as the cloning issue has been resolved.

RESOLVED: Intermittent my.pressable.com Control Panel Downtime

Howdy,

We created a previous post about our email provider outage potentially causing issues with email sending from our servers but wanted to specifically point out that this also affects the https://my.pressable.com control panel.

Users are seeing the control panel time out or be completely unresponsive on an intermittent basis.

These issues are also related to the same issues that are caupring problems with email in general because of the ties we in place for email on my.pressable.com.

As soon as Mailgun has these issues resolved, the control panel will function properly once more.

You can follow the status of the Mailgun issues here:

http://status.mailgun.com/incidents/9s93dc83gtlw

If you have quesitons or concerns, please send an email to help@pressable.com.

Thank you!

UPDATE 09/30/2014 @ 12:40 PM Central: Email issues with all sites have been resolved. http://status.pressable.com/2014/09/29/email-provider-outage-causing-issues/

RESOLVED: Email Provider Outage Causing Issues

Howdy,

Our email provider, Mailgun, is experiencing issues that is causing our customer emails to not be sent in some cases. This originally resulted in emails from sites not being sent appropriately in addition to ones that come from our https://my.pressable.com control panel.

These issues occur as a result of a larger emergency maintenance at Rackspace itself.

As this maintenance has progressed, our email provider has subsequently been able to restore service and emails from sites have begun sending on a much more consistent basis.

As of this morning, Mailgun is still working through some lingering issues and one of those issues affects emails being sent from our https://my.pressable.com control panel.

Once they are done addressing this, emails will work appropriately across our network again.

You can read more about the Mailgun outage here:

http://status.mailgun.com/incidents/9s93dc83gtlw

You can read some about the larger Rackspace Emergency Maintenance here:

https://status.rackspace.com/

If you have any questions or concerns, please contact our helpdesk at help@pressable.com.

UPDATE 09/29/2014 @ 11:07 AM Central: Our https://my.pressable.com control panel is experiencing intermittent downtime as a part of this ongoing issue. If you are having trouble accessing the control panel, you might try giving it a few minutes and trying again. Once this issue has subsided, the panel will work normally again.

UPDATE 09/30/2014 @ 6:30 AM Central: We’re still working with our email provider (Mailgun) to resolve lingering issues caused by their downtime. Currently issues seem to only be impacting customers in the same datacenter as Mailgun services. We’ll provide an update as soon as we’ve learned more from our provider.

UPDATE 09/30/2014 @ 11:00 AM Central: We’ve confirmed with our provider (Mailgun), that they are still experiencing issues with their systems. We’re in constant communication with them providing them details as they test fixes to bring services back to 100%. We’ll update you as we have more information.

UPDATE 09/30/2014 @ 12:40 PM Central: At this time mail services have been restored across our systems. Our tests show everything working properly now and we’re expecting no further issues with email delivery. A huge thanks to Rackspace and Mailgun for staying on top of this and getting things resolved.

[RESOLVED] Slave Database Running Behind

Howdy,

We are currently experiencing an issue with one of our slave database servers that is causing some sites to see out of date information and load slower than normal. This issue is also causing some changes and edits to sites to appear to not be saving properly.

We are working on getting this slave database caught back up and will update this with more information as soon as this has been completed.

If you have any questions or concerns, please submit a help desk ticket via my.pressable.com.

Please let me know if you have any questions, thanks!

UPDATE 08/19/2014 @ 7:00PM Central: The database server that was falling behind has caught up and items served from this database are now serving properly.

[RESOLVED] Issue With Slow and Improperly Loading Sites

Howdy,

We are currently addressing an issue that is causing some sites to load very slowly and not at all in some cases. Some sites are also seeing a “Coming Soon” Pressable page.

We are addressing this issue now and will update here once the issue is resolved.

If you have any questions or concerns, please submit a ticket via my.pressable.com

UPDATE 08/13/2014 @ 11:45 AM Central: Sites that are/were affected by this are now loading but are still a little a slow in some cases. We are continuing to address this issue where it is present and will update with further information as it becomes available.

UPDATE 08/13/2014 @ 1:15 PM Central: The team is continuing to address issues with slow and improperly loading sites.

UPDATE 08/13/2014 @ 2:20 PM Central: The team is continuing to address issues with slow and improperly loading sites.

UPDATE 08/14/2014 @ 1:45 PM Central: All servers are now stable and running properly. We have made many improvements that will help prevent similar issues going forward. We will be making a blog post soon that details these improvements and explaining the issues as they occurred shortly. If you have questions or concerns, please contact us via my.pressable.com if you have questions or concerns. Thank you for your patience and support, our team truly appreciates it!

RESOLVED: Instability on Web Clusters

Howdy,

We are experiencing suspicious activity that is causing instability on our some of our web server clusters.

We are working on expanding the stability/performance on this cluster and identifying the source of this suspicious activity. During this time, customers may experience intermittent 502s and slow loading sites.

We will update the status blog as soon as this is finalized.

UPDATE 2:50PM CST: The team is still working to bring new hardware and firewall rules online to help mitigate the attacks on our systems. We’ve brought new hardware online for some customers which is beginning to return some stability to the systems.

UPDATE 4:00PM CST: At this time we’ve brought on our new hardware and firewall rules for all of the customers on Thor. We’re still working to get these in place for our customers on Galaxy01.

UPDATE 5:55PM CST: At this time we’ve brought up new hardware and firewall rules for all customers. Stability should be returning to the web servers, but we’re still experiencing database issues.

RESOLVED: Issue Causing Old Data To Display On Some Sites

We are experiencing an issue that is causing some sites on the network to show/display old data. This same issue is also causing some data to show as not saving properly.

We are working on addressing this issue and will update the status blog as soon as things have returned to normal.

If you have questions or concerns, please submit a ticket via my.pressable.com, thanks!

UPDATE 7:15PM CST: The issues causing some database servers to return older (stale) data has been corrected and all sites should be functioning normally now.