September 1st (Labor Day) Support Hours

Howdy,

September 1st, 2014 is a holiday in the USA  and we will be operating with a skeleton crew throughout the day so that we can observe the holiday with our family and friends.

A skeleton crew means that we will have a limited/reduced number of staff members on call to answer tickets as they come through and as the workload permits. We will also be responding to any and all emergencies that may arise to ensure stability and reliability through the holiday.

If you have questions or concerns, please submit a ticket through my.pressable.com and we can address these accordingly.

The Pressable staff would like to thank all of our customers for their support this year and through the last few months. We are constantly appreciative of the feedback we receive and are looking forward to serving you in the future. Have a wonderful holiday!

[RESOLVED] Slave Database Running Behind

Howdy,

We are currently experiencing an issue with one of our slave database servers that is causing some sites to see out of date information and load slower than normal. This issue is also causing some changes and edits to sites to appear to not be saving properly.

We are working on getting this slave database caught back up and will update this with more information as soon as this has been completed.

If you have any questions or concerns, please submit a help desk ticket via my.pressable.com.

Please let me know if you have any questions, thanks!

UPDATE 08/19/2014 @ 7:00PM Central: The database server that was falling behind has caught up and items served from this database are now serving properly.

[RESOLVED] Issue With Slow and Improperly Loading Sites

Howdy,

We are currently addressing an issue that is causing some sites to load very slowly and not at all in some cases. Some sites are also seeing a “Coming Soon” Pressable page.

We are addressing this issue now and will update here once the issue is resolved.

If you have any questions or concerns, please submit a ticket via my.pressable.com

UPDATE 08/13/2014 @ 11:45 AM Central: Sites that are/were affected by this are now loading but are still a little a slow in some cases. We are continuing to address this issue where it is present and will update with further information as it becomes available.

UPDATE 08/13/2014 @ 1:15 PM Central: The team is continuing to address issues with slow and improperly loading sites.

UPDATE 08/13/2014 @ 2:20 PM Central: The team is continuing to address issues with slow and improperly loading sites.

UPDATE 08/14/2014 @ 1:45 PM Central: All servers are now stable and running properly. We have made many improvements that will help prevent similar issues going forward. We will be making a blog post soon that details these improvements and explaining the issues as they occurred shortly. If you have questions or concerns, please contact us via my.pressable.com if you have questions or concerns. Thank you for your patience and support, our team truly appreciates it!

RESOLVED: Instability on Web Clusters

Howdy,

We are experiencing suspicious activity that is causing instability on our some of our web server clusters.

We are working on expanding the stability/performance on this cluster and identifying the source of this suspicious activity. During this time, customers may experience intermittent 502s and slow loading sites.

We will update the status blog as soon as this is finalized.

UPDATE 2:50PM CST: The team is still working to bring new hardware and firewall rules online to help mitigate the attacks on our systems. We’ve brought new hardware online for some customers which is beginning to return some stability to the systems.

UPDATE 4:00PM CST: At this time we’ve brought on our new hardware and firewall rules for all of the customers on Thor. We’re still working to get these in place for our customers on Galaxy01.

UPDATE 5:55PM CST: At this time we’ve brought up new hardware and firewall rules for all customers. Stability should be returning to the web servers, but we’re still experiencing database issues.

RESOLVED: Service Interruption

We’re currently investigating an issue with site availability and system stability. We’ll provide an update as we have more details.

UPDATE 12:55PM CST: The team has identified an issue with one of the database servers and is currently evaluating the system.

UPDATE 1:20PM CST: The team is currently looking at an issue with one of the physical drives in the database server.

UPDATE 2:00PM CST: At this time we’ve restored basic services to sites. The team is still working to get the database servers back to 100%, but sites should be operational.

NOTICE: For the next several hours some of the database slaves will be serving stale data. This will occur while our team works to bring up new slave database servers. If you’re having issues with new content not showing up on your site, this is why. If you’re working to publish new content, and having problems, please contact help@pressable.com and our team can assist.

UPDATE 5:00PM CST: Our team is investigating load issues from the earlier database problems. This is causing sites to be down and unresponsive.

UPDATE 10:00PM CST: Site stability has returned, however, we’re still waiting for the database system to return to 100%.

IMPORTANT NOTICE: For the next several hours some of the database slaves will be serving stale data. This will occur while our team works to bring up new slave database servers. If you’re having issues with new content not showing up on your site, this is why. If you’re working to publish new content, and having problems, please contact help@pressable.com and our team can assist.

UPDATE 8/10/14 7:50AM: We’re still working to get the databases servers back to 100%. During this time database slaves will be serving stale data. This will occur while our team works to bring up new slave database servers. If you’re having issues with new content not showing up on your site, plugin changes not saving, or comments not posting it’s related to this issue. If you need to publish new content or make updates, please contact help@pressable.com and our team can assist.

UPDATE 8/10/14 12:33 PM: We are continuing to work on the database servers. During this time, we have put some changes into place on our Thor cluster that will help improve stability and performance. You may experience a 502 during this time and this will clear once all changes have been finalized. Please contact us through my.pressable.com if you have questions or concerns.

UPDATE 8/10/14 4:25PM: Just as a reminder, we’re not out of the woods with the database issues. We’re still working to get back to 100% and are dealing with stale data and sites not updating. If you’re having issues with new content not showing up on your site, plugin changes not saving, or comments not posting it’s related to this issue. If you need to publish new content or make updates, please contact help@pressable.com and our team can assist.

UPDATE 8/10/14 6:00PM: We’re still working to get back to 100% and are dealing with stale data and sites not updating. If you’re having issues with new content not showing up on your site, plugin changes not saving, or comments not posting it’s related to this issue. If you need to publish new content or make updates, please contact help@pressable.com and our team can assist.

UPDATE 8/10/14 7:20PM: At this time we’ve restored two new database slaves and put them in place for all customers. We’re continuing to monitor the database servers and make sure that things work properly, but early signs are positive. If you’re still having issues, please email help@pressable.com and we’ll take a look.

UPDATE 8/11/14 8:55AM: Overnight we noticed that one of the new servers we put in place wasn’t able to keep up with the demand of traffic. This caused momentary issues where it would serve stale data while other servers would serve the latest data. We’ve now replaced this machine and stability has returned to the systems. If you’re still having issues, please email help@pressable.com and we’ll take a look.

RESOLVED: Issue Causing Old Data To Display On Some Sites

We are experiencing an issue that is causing some sites on the network to show/display old data. This same issue is also causing some data to show as not saving properly.

We are working on addressing this issue and will update the status blog as soon as things have returned to normal.

If you have questions or concerns, please submit a ticket via my.pressable.com, thanks!

UPDATE 7:15PM CST: The issues causing some database servers to return older (stale) data has been corrected and all sites should be functioning normally now.

RESOLVED: Potential Mail Delivery Issue

We are experiencing issues with outbound email notifications from our systems. This means password reset, comment notification, registration, and other similiar types of emails are not being delivered. We are working with our email delivery provider SendGrid, to resolve this issue.

If you have any questions or concerns, please submit a ticket for through my.pressable.com

Thank you!

UPDATE 08/01/2014 @ 8:46 AM CST: We are still working on clearing this issue up. Email that is coming from websites is still not delivering properly and we are continuing to work toward resolution.

UPDATE 08/01/2014 @ 3:47 PM CST: We have notified SendGrid and asked them to make changes on their end that will get email flowing again. We will update here as soon as mail is back up and running.

UPDATE 08/02/2014 @ 10:00 PM CST: We have still been unable to get email service restored and are continuing on working toward a solution here. We apologize for the continued issues for those who are affected by this and will update again once further progress is made.

UPDATE 08/04/2014 @ 1:15 PM CST: At this time email services have been restored across our systems. If your site is still having issues sending email, please contact us so that we may investigate. Apologies for the delay in getting this resolved.

502 Errors on Galaxy01 Cluster

We had a brief issue with out Galaxy01 cluster that caused sites on these servers to send back a 502. We have addressed this issue and everything is now working as expected.

If you continue to experience issues, please let us know by submitting a ticket through your my.pressable.com panel.

Thank you!

Outage – Unresponsive Servers

Howdy,

We are currently experiencing an issue with a few servers that are not responding properly. This is causing sites on these clusters to not load properly or at all in some cases. We are working swiftly to bring these servers back online and resolving this issue.

If you have any questions, please submit a ticket to our help desk via http://my.pressable.com

UPDATE July 17th, 2014 at 3:01 PM Central:  Most customers are seeing sites back up and running by now. We have more servers coming back up as this is posted and service will be restored entirely to all sites shortly.

UPDATE July 17th, 2014 at 4:01 PM Central: All sites have been restored to full service by now. If for any reason you are still experiencing issues with a site being down, please let us know and we can address it accordingly.

UPDATE July 17th, 2014 at 5:30 PM Central: We are experiencing issues on our “thor” cluster still that is causing some sites to be very slow. We are working to restore sites on this cluster back to full speed and functionality.

UPDATE July 17th, 2014 at 5:30 PM Central: All servers in the “thor” cluster are now back in full functioning order. Sites will now run at their expected speed now. Thanks for your continued patience while we resolved issues here.

4th of July Holiday Support Hours

Howdy,

In observance of the holiday on July 4th, 2014, we will have a reduced support staff on hand and will be operating with minimal support response. We will resume operating under normal support hours on July 5th, 2014.

Minimal Support Response means that we won’t be available to respond to non-emergency requests. There will be support staff standing by, but we use our best judgement to determine emergency issues that need our attention immediately.

We hope everyone enjoys their holiday and look forward to serving you through the rest of the year.

Thank you!