Database Issue Impacting Site Availability

We’re currently experiencing an issue impacting one of our primary database servers. This issue is impacting the availability of sites for many customers and we’re actively working to troubleshoot the issue. We’ll provide an update when we have more information.

UPDATE April 3, 12:00pm CST: We’re still investigating issues with the database servers causing slow sites and site unavailability.

SFTP Outage In Dallas Datacenter

We are currently experiencing an SFTP outage in our Dallas datacenter that is causing files to be inaccessible via SFTP.

All sites are up and functioning normally as this does not affect the site running itself, only access to site files via SFTP.

We are working on a fix and will update the status blog when this has been resolved.

If you have any questions or concerns, please submit a ticket to our help desk via your my.pressable.com panel.

Update (5:01 PM Central): Service has now been restored and SFTP is functioning normally again.

Slow Sites and Intermittent Downtime

We are experiencing an issue that is causing some sites on our system to load slowly and/or experience intermittent downtime.

We are working closely with Rackspace to figure out what is causing this and to restore performance back to normal.

Please check back here for updates as they become available.

If you have questions or concerns, please contact us by submitting a help request through your https://my.pressable.com panel.

Intermittent DDOS attacks at one of our data centers

Hi Everyone,

Over the past 72 hours, one of our data centers at Rackspace has been the target of a DDOS (Distributed Denial of Service) attack. Due to this attack, some of our customer websites were affected as well. We are working with Rackspace to mitigate this issue.  Here is the latest communication from our contact at the NetSec team at Rackspace.

"Hello,

On 19 February 2014, at approximately 15:28 UTC / 09:28 CST, a portion of Rackspace customers may have encountered network related issues as a result of a DDoS (Distributed Denial of Service) attack targeted at an IP address within our infrastructure. The DDoS was identified by our network team, who resolved the issue at 15:31 UTC / 09:31 CST by null routing the target IP address. During this time, you may have experienced latency and or packet loss.

We apologize for any inconvenience this caused and appreciate your patience as we worked to resolve the issue as quickly as possible. If you have any further questions or concerns, please contact a member of your support team.
"

 

WordPress 3.8 Updates Rolling Out

We’ve been closely monitoring the WordPress 3.8 release and how sites have been handling the update. We’re happy to report that at this time we’re going to begin rolling out this new update to all sites.

You can read more about the WordPress 3.8 release on WordPress.org and on The Pressable Blog.

If you have any questions about the update, please feel free to contact the support team through https://my.pressable.com.

RESOLVED: SFTP Service Currently Unavailable

We’re currently investigating an issue that’s impacting the availability of the SFTP service. This issue appears to be caused by a higher than normal number of connections to the system. We’re working to restore services as quickly as possible.

We’ll provide an update to this post when more information becomes available.

UPDATE 9:45am CST: At this time SFTP services have been restored. We’ve isolated the issue with connections to the system and put in new measures to monitor these and prevent further issue. If you’re still experiencing issues, or if you have any questions, please let us know.

RESOLVED: Email Outage Across All Systems

On December 30th, 2013 there was a system wide email outage that impacted delivery of email from all of our systems. This includes customer WordPress sites, notification emails from My Pressable, support requests submitted through My Pressable, and email inquiries through our own website.

What Happened: On December 27th we were notified through some older (zippykid.com) channels of an issue with our account at our provider. On December 30th there was a miscommunication with a configuration change and our account was temporarily suspended. This suspension resulted in the immediate halting of all email sent from our systems.

We were able to identify and correlate the issue through several customer issues. Once we determined the issue was with the changes at our provider we worked with them to restore services. At approximately 5:00pm CST our account was restored, and messages began to queue for delivery. However, complete services were not restored until 9:00pm CST.

What We’re Doing: We’ve made some adjustments to the account control for this service provider so that more people are aware of changes made. We’ve also evaluated fallback options should we need to implement a new solution in the future.

While this was a unique issue, we do apologize for the scope and time of the outage. We’re continuing to see what else we may be able to do to improve the reliability of this in the future as well as prevent any further issues.

Please also keep in mind, that if a request for support was submitted through the My Pressable control panel during this time, we would NOT have received that request. We apologize for that, but please re-submit your issue if you still require assistance.

If you have any questions, please feel free to contact our support team.

Winter Holiday Support Hours

Wow… 2013 sure has flown by! It’s been a busy year for all of us at Pressable and it’s time to wind down, celebrate with friends and family, and come back in 2014 with new and exciting things.

During the holiday season, we’ll have reduced support beginning on Tuesday, December 24 through December 27. We’ll also have minimal support Tuesday, December 31 through Wednesday, January 1.

What this means for you:

  • December 24: Emergency Support Only
  • December 25: Emergency Support Only
  • December 26: Minimal Support Response
  • December 27: Minimal Support Response

For the remainder of the holiday weekend, support staff will be limited in availability. This means response times may be slower, or some non-critical items may be held until the following Monday.

  • December 31: Minimal Evening Support Response
  • January 1: Emergency Support Only

Emergency Support means that we’ll only be available for system wide outages/issues impacting the customer base at large.

Minimal Support Response means that we won’t be available to respond to non-emergency requests. There will be support staff standing by, but we use our best judgement to determine emergency issues that need our attention immediately.

We apologize in advance for any inconvenience this may cause you. If you have questions please feel free to direct them to help@pressable.com.

COMPLETED: Scheduled Database Maintenance – Sunday December 15th at 11:00pm CST

On Sunday, December 15th at 11:00pm CST we’ll be performing maintenance on our database servers that will cause sites to be unavailable for a period of time. During the maintenance sites will show the following message:

“Error establishing a database connection”

This maintenance is expected to last no more than 30 minutes, but we’re hopeful things will be completed much quicker. Please also note, this maintenance will only impact customers in our “ibm.dal” datacenter. Customers can quickly identify if they’re in this datacenter by looking at their sftp hostname, or checking to see if their site uses a CNAME containing soyuz, sputnik or pagespeed-beta.

We will also be notifying impacted customers via email. If you have any questions, please don’t hesitate to contact us through the http://my.pressable.com control panel.

UPDATE 11:45PM CST: At this time we’re still working to complete the maintenance. Things should be wrapped up shortly as things are still progressing as planned.

UPDATE 12:00AM CST: The maintenance has been completed as planned and sites should be functioning normally. We appreciate your patience during this, and if you’re experiencing any questions, please don’t hesitate to contact us.