No News Is Good News

We only post on this site when something that might affect Birdhouse services has occurred, or when we’re expecting a planned outage.  Sometimes we go for many months  without posting here. That’s a good thing – it means things are running smoothly!

Everything below this post refers to a service-affecting event.

 

Datacenter Outage

The Birdhouse datacenter is experiencing a severe upstream outage which is currently preventing all access. We don’t have more information at the moment but will post again when we do.

Five minutes later… Well that didn’t last long. The issue has been corrected and we’re back online. Thanks for your patience.

Update: Servint’s excellent blog post on the difficulty of diagnosing problems when routers get schizophrenic about their own health status.

Conversion to Dovecot IMAP/POP Server

To address  performance issues  related to IMAP connections coming from OS X Mail.app users (we have many of them!), we have switched the mail server software at Birdhouse from Courier to Dovecot. We expect that this change will allow Mail.app to connect to mail services more reliably.

Users may need to re-accept the SSL certificate for gong.birdhouse.org.

Some users will find some duplicate emails appear in their mail clients. This issue will primarily affect anyone still using POP3 rather than IMAP for mail checking.

Please let us know if you have questions about this change, or if you notice performance changes for better or for worse.

 

MySQL 5.5 Upgrade

We’ve upgraded all MySQL instances from version 5.0 to 5.5, with an intermediate 5.1 upgrade in the middle. All WordPress and Django sites appear to be running normally after the upgrade – please let us know if you experience any issues.

Scheduled Downtime, October 20

Birdhouse Hosting will be down for maintenance the evening of Saturday 10/20, starting around 5pm Pacific / 8pm EST. The outage will affect all web and mail services. The length of the downtime is indeterminate (depending on how long the datacenter takes to do their work), but is estimated to last between 1 and 8 hours  – almost certainly on the short side of that window.

All inbound mail will be re-tried by sending servers and will arrive safely once we’re back online.

Conversion to PHPSuExec

This weekend we’ll be changing the way customer domains interact with PHP on the main web server. After we make this change, rather than everyone sharing a common PHP process, each PHP request will be owned by the user who owns the files in that directory.

This technique, known as PHPSuExec, has benefits for both you the customer and us as system administrators. For you, it means you’ll be able to use WordPress’ built-in updater for core, plugins, and themes (among other benefits). For us, it means we’ll be able to see at a glance which Apache processes are consuming a lot of resources – we’ll be able to know immediately who has the busy sites. This will greatly add in trouble shooting.

And for all of us, it means more security.

Your site may experience a brief bit of downtime while this conversion happens. We’ll manually check all sites after the upgrade it complete to make sure all sites are working properly.

Planned downtime

We are back online as of early this morning, but to ensure that we are not at risk of further outages from disk failure, we have opted to move our hosting environment to a different physical server in the same datacenter. This will work will take place between midnight and 3am, PST tonight.

Customers do not need to change anything, and no data or configurations will be affected.

Thanks for your patience.

Update: The migration is unfortunately taking far longer than expected. Nothing has gone wrong, but the engineer who provided the original time estimate was unfortunately way off. We apologize for the inconvenience.

Update 2: All systems came back online around 9:00 am PST.