About the recent downtime (2010-09-24 and 25)

You certainly have noticed that trac-hacks.org was down for more than 24 hours. I apologize for any inconvenience caused by this incident. Here is what happened:

trac-hacks.org is hosted as a virtual server (based on ​linux-vserver) on the physical server of the ​MadWifi project. That server suffered from an issue that caused the host server to more or less hang. Our monitoring notified me of the problems by SMS. At that time I was on a holiday trip without internet access, so I asked my "remote hands" to reboot the server. That indeed helped to fix the original issue and the monitoring reported the server to be up and running again. Unfortunately the trac-hacks.org vserver encountered an issue during startup, preventing it from coming up normally, which remained unnoticed by the monitoring - and me. I only noticed it when I came online this evening after returning home.

On the positive side, this incident pointed out how to improve the monitoring and the startup process of the hosted vservers to prevent these issues in the future. These things will be taken care of asap.

Comments

No comments.