rss
Downtime 2011-04-06/07
The server that hosts trac-hacks.org along with the websites of the MadWiFi project has been suffering from a kind of hardware malfunction. It became unresponsive for short timespan first, then worked for several hours without any signs of problems. Things became pretty borked during this night, then at 04:22 UTC+2 the server failed. Power cycling plus a little prayer :-) helped to finally fix it at 07:00 UTC+2.
About the recent downtime (2010-09-24 and 25)
You certainly have noticed that trac-hacks.org was down for more than 24 hours. I apologize for any inconvenience caused by this incident. Here is what happened:
trac-hacks.org is hosted as a virtual server (based on linux-vserver) on the physical server of the MadWifi project. That server suffered from an issue that caused the host server to more or less hang. Our monitoring notified me of the problems by SMS. At that time I was on a holiday trip without internet access, so I asked my "remote hands" to reboot the server. That indeed helped to fix the original issue and the monitoring reported the server to be up and running again. Unfortunately the trac-hacks.org vserver encountered an issue during startup, preventing it from coming up normally, which remained unnoticed by the monitoring - and me. I only noticed it when I came online this evening after returning home.
On the positive side, this incident pointed out how to improve the monitoring and the startup process of the hosted vservers to prevent these issues in the future. These things will be taken care of asap.
Downtime on 2009-06-04 [updated]
trac-hacks.org will be down for some hours tomorrow, beginning at about 2009-06-04 06:00am UTC+2 (see here for your local time). It will be moved to another, more powerful server. Yes, unless anything unforeseeable happens this should be the last move required for t-h.o for some time. :-)
Update:
The work started on 07:00am, with a one hour delay. Things went smooth, so the website went online on 07:50am on the new server. Thanks for your patience.
10 minutes downtime on 2009-03-24, 5 to 6am UTC+1
trac-hacks.org will be offline for approximately 10 minutes on March 24, 2009 sometimes between 5 and 6am, UTC+1 (see here what that is in your local time). A central switch is going to be replaced, which affects the connectivity of the rack our server is in.
