We will be upgrading our database server this Thursday (6th September 07). We expect this exercise to take anywhere between 2 to 5 hours.

The forums will be taken offline from 1030pm to 12am (and probably be inaccessible up till 5am depending on any complications that might happen during the transfer).

The new database server should be able to cope with the peak hour load that has been causing the recent increase in "server busy" messages.

So far, the average-case timings I have are:

15 minutes - mysqldump
5 minutes - transfer
15 minutes - mysql import

Plus another 20 minutes for hardware fiddling. I think 2 hours is a reasonable window to expect

The new server doesn't seem to be coping very well. We're busy trying to figure out why it randomly puts itself to sleep!

---

Added on September 8, 2007, 7:51 pmServer has been stable for last 24+ hours without any signs of problems.

We'll continue to monitor and hopefully everything is ok.

Here's some performance graphs before and after the upgrade.


This is the old DB server's typical load over a day. Look at the fat IO Wait graph, indicating that we're IO-saturated.


While this is the new DB server's typical load. What load!?


And as a consequence of reduced DB backlog, the webserver's load spikes have reduced as well. This is a weekly graph, so compare the first 3 days (pre-upgrade) and the days following it.