Sporadic Functionality of Deploys and Clones

We are currently experiencing an issue that is causing clones and new site deploys to function sporadically.

Some customers are experiencing issues cloning sites and reporting unfinished clones. New site deploys are functioning for a large majority of our customers but there are still cases where new site deploys are not working.

We are attempting to resolve these issues as quickly as possible and will update this status blog as more information comes available.

If you have questions or concerns, please contact us by submitting a help desk request via your https://my.pressable.com control panel.

UPDATE Nov. 21, 2014 @ 10:41AM CST: Deploys across our systems are now working properly effective immediately. We are still working on fixing cloning across the board. We discovered that clones broke partially due to a fix we put in for POODLE recently. We will update again once this issue is resolved.

UPDATE Nov. 25, 2014 @ 9:20AM CST: Deploy functionality continues to work properly across our systems. The team is still working to resolve the incompatibilities with our cloning functionality and the increased security put in place post-POODLE.

UPDATE Dec. 1, 2014 @ 10:38AM CST: We are continuing to work on fixing the cloning issues from this past week. We have removed all clones that failed while this has been happening. We are asking for customers to refrain from attempting to clone until this issue has been patched fully. We will update again as soon as the cloning issue has been resolved.

Resolved – Chicago DC – Unscheduled Maintenance October 25th 2014

Due to a miscommunication between us and our service provider Rackspace, the majority of websites hosted at our Chicago datacenter are offline.  We are on the phone with Rackspace working to restore the servers that are needed to bring these websites online as quickly as possible. 

As of Saturday October 25th 8:31AM. All services are running properly.

POODLE SSL Vulnerability

We are aware of the recently announced vulnerability in SSL called POODLE. This vulnerability affects a very small percentage of traffic on our systems.

You can read more about this vulnerability at the following links:

https://isc.sans.edu/diary/OpenSSL%3A+SSLv3+POODLE+Vulnerability+Official+Release/18827

Click to access ssl-poodle.pdf

The “fix” for this issue is to disable SSLv3 on servers and/or clients that utilize it. Fortunately, it is in use in a very small percentage of traffic and is not likely to hinder traffic to sites we host in any significant capacity.

The recommended solution to this problem will make clients using Internet Explorer 6 running on Windows XP or older recieve an SSL error.

Currently, this fix will also break the Pingdom monitoring service SSL check. Pingdom is aware of this issue and is making the appropriate changes to their systems.

We are aware of the inconvenience this may cause, but we believe that this is the right thing to do.

You can see larger providers implement similar policies here.

https://chargify.com/blog/dropping-sslv3/
https://blog.cloudflare.com/sslv3-support-disabled-by-default-due-to-vulnerability/

If you have any questions or concerns, please contact our help desk by submitting a ticket from https://my.pressable.com.

RESOLVED: Intermittent my.pressable.com Control Panel Downtime

Howdy,

We created a previous post about our email provider outage potentially causing issues with email sending from our servers but wanted to specifically point out that this also affects the https://my.pressable.com control panel.

Users are seeing the control panel time out or be completely unresponsive on an intermittent basis.

These issues are also related to the same issues that are caupring problems with email in general because of the ties we in place for email on my.pressable.com.

As soon as Mailgun has these issues resolved, the control panel will function properly once more.

You can follow the status of the Mailgun issues here:

http://status.mailgun.com/incidents/9s93dc83gtlw

If you have quesitons or concerns, please send an email to help@pressable.com.

Thank you!

UPDATE 09/30/2014 @ 12:40 PM Central: Email issues with all sites have been resolved. http://status.pressable.com/2014/09/29/email-provider-outage-causing-issues/

RESOLVED: Email Provider Outage Causing Issues

Howdy,

Our email provider, Mailgun, is experiencing issues that is causing our customer emails to not be sent in some cases. This originally resulted in emails from sites not being sent appropriately in addition to ones that come from our https://my.pressable.com control panel.

These issues occur as a result of a larger emergency maintenance at Rackspace itself.

As this maintenance has progressed, our email provider has subsequently been able to restore service and emails from sites have begun sending on a much more consistent basis.

As of this morning, Mailgun is still working through some lingering issues and one of those issues affects emails being sent from our https://my.pressable.com control panel.

Once they are done addressing this, emails will work appropriately across our network again.

You can read more about the Mailgun outage here:

http://status.mailgun.com/incidents/9s93dc83gtlw

You can read some about the larger Rackspace Emergency Maintenance here:

https://status.rackspace.com/

If you have any questions or concerns, please contact our helpdesk at help@pressable.com.

UPDATE 09/29/2014 @ 11:07 AM Central: Our https://my.pressable.com control panel is experiencing intermittent downtime as a part of this ongoing issue. If you are having trouble accessing the control panel, you might try giving it a few minutes and trying again. Once this issue has subsided, the panel will work normally again.

UPDATE 09/30/2014 @ 6:30 AM Central: We’re still working with our email provider (Mailgun) to resolve lingering issues caused by their downtime. Currently issues seem to only be impacting customers in the same datacenter as Mailgun services. We’ll provide an update as soon as we’ve learned more from our provider.

UPDATE 09/30/2014 @ 11:00 AM Central: We’ve confirmed with our provider (Mailgun), that they are still experiencing issues with their systems. We’re in constant communication with them providing them details as they test fixes to bring services back to 100%. We’ll update you as we have more information.

UPDATE 09/30/2014 @ 12:40 PM Central: At this time mail services have been restored across our systems. Our tests show everything working properly now and we’re expecting no further issues with email delivery. A huge thanks to Rackspace and Mailgun for staying on top of this and getting things resolved.

Slave Databases Running Behind

Howdy,

We are currently experiencing an issues with our slave database servers that is causing some sites in our network to serve stale content. We are aware of these issues and are working toward resolving them now.

This is not currently causing sites to be unavailable, just serve old content in some cases.

If you have any questions or concerns, please contact our help desk by submitting a ticket via your https://my.pressable.com control panel.

Slider Revolution Plugin Security Vulnerability

IMPORTANT: Sucuri has released details on an exploit for the plugin Slider Revolution Responsive WordPress Plugin. Details of the exploit can be seen on Sucuri’s blog.

IF YOU ARE USING THIS PLUGIN, PLEASE UPDATE TO THE LATEST VERSION IMMEDIATELY.

Our team is working on a way to block these attacks, but we can NOT update this plugin on sites as it’s not freely available on the WordPress.org repository.

September 1st (Labor Day) Support Hours

Howdy,

September 1st, 2014 is a holiday in the USA  and we will be operating with a skeleton crew throughout the day so that we can observe the holiday with our family and friends.

A skeleton crew means that we will have a limited/reduced number of staff members on call to answer tickets as they come through and as the workload permits. We will also be responding to any and all emergencies that may arise to ensure stability and reliability through the holiday.

If you have questions or concerns, please submit a ticket through my.pressable.com and we can address these accordingly.

The Pressable staff would like to thank all of our customers for their support this year and through the last few months. We are constantly appreciative of the feedback we receive and are looking forward to serving you in the future. Have a wonderful holiday!

[RESOLVED] Slave Database Running Behind

Howdy,

We are currently experiencing an issue with one of our slave database servers that is causing some sites to see out of date information and load slower than normal. This issue is also causing some changes and edits to sites to appear to not be saving properly.

We are working on getting this slave database caught back up and will update this with more information as soon as this has been completed.

If you have any questions or concerns, please submit a help desk ticket via my.pressable.com.

Please let me know if you have any questions, thanks!

UPDATE 08/19/2014 @ 7:00PM Central: The database server that was falling behind has caught up and items served from this database are now serving properly.