Phew, it took me three goes to get that posted! After moving to the new Root Server I was having problems with some of the scripts that automatically create files somewhere in the document root.

To get this working I just decided to run Apache under the same user as the owner of the document root. This then caused the problem that all the session variables were not being saved! Quick chown to the new user on the temp directory used to save the session files sorted that.

Cluster Update: I´ve gone ahead and started building my Linux cluster to act as my way-over-the-top web server. I grabbed 6 identicle Athlon based systems that I wasn´t using and ripped out the hard disks (I only need the drive in the master node). These were then all networked and I then went searching for tutorials on how to set it all up.

After a bit of searching I found this tutorial which is quite good. I used Fedora Core 3 since I already had the disks on my desk but it works the same. The only bit that I found confusing was adding the libraries lines which after more searching is added to the file /etc/ld.so.conf. I´ve booted it all up with the master and one slave node and I´m about to do a boot with all 5 slaves. After that I might do some testing to see what kind of performance I´m getting out of it.

I was reading an article in a PC mag entitled "Build your own supercomputer" but as it turns out it was mainly supercomputer history and then talking about basic multiple core PCs which are more powerful than a $10 million Cray from the early 90s. The end of the article though was a lot more interesting. It was about using bog standard PCs to create a linux cluster to do some serious number crunching.
 
All you really need is a bunch of old PCs, a switch or two and the software to run it all. The PCs can be pretty much anything as long as they have a COU, RAM and a network card. The next step is to make sure they are linked up to the switch or switches to optimise the communication speeds between the different boxes. After that it´s just a case of installing the software (all of which is free to download since it´s open source) and configuring the entire system.

 
I also want mine to be able to run with one machine on a UPS and set up so that if one or all of the others go down (either due to power failure or hardware failure) it will keep running as a system and when the other systems come back on line they will just boot and join the cluster again. I´ve got a lot more Googling to do to find out how to do all this and if all of it is possible but I´ll post more here when it gets done

After all these years of having nice wizards and configuration menus to help you set up your servers, Microsoft is releasing a server version of Windows that comes with no GUI, just a command line. Time to join us Linux sysadmins in doing it the good old fashioned way.
 
Trust me, its more fun this way anyway and you learn more about how things work. Plus the non-technical people around you think you´re doing something really complicated!

After weeks of planning, massive reconfigurations, new backup scripts and file moves, I think I´m finally ready to move servers!
 
I´ve just to figure out a night when we should be fairly quiet to do the move so the sites aren´t down too long. I´ve got it all planned out and should be able to do it in stages so that I don´t have to do a series of all-nighters. Plus it is the holiday season so the distruption to the public should be minimal.
 

Just got to wait until Tuesday for a test script to run successfully and then I can go ahead and do it!

We´ve got a system that gets used for most of our online projects that allows companies to sign up for a service and then automatically creates a pdf invoice to send them. This then gets downloaded by the office server and then printed out once an hour.
 
We were going through the last days invoices and noticed that there was a big gap in invoice numbers. I checked the database and found that a load hadn´t been printed. This was probably because the power had been off so the download server was running but the print server was still turned off so they were just disappearing into nowhere.
 
Luckily we have another networked printer that I can use and if the download server can´t directly contact it, it will keep it in its print queue until it comes back up again. Sorted. Just got to make sure we don´t run out of toner or nothing´ll get printed until we get more.

Well, I´ve got around to setting up the development servers in the office, distributing the development files and databases across them and set up the backup routines. I´ve also changes all the files to take into account the server move so all I need to do now is set up the new servers, move accross and test.

 
The move and testing are pretty simple since it´ll just require taking down the database at a certain point, moving the files accross and bringing it live again. As long as I do this at stupid-o-clock then nobody should loose any info or even notice.
 
The main problem is the DNS shift. If I was moving to another host then this would take 24 hours and result in serious downtime, but since the same DNS servers are being used and there is just an IP address change it should go through a lot quicker. I´ll still do it at night to minimise disruption though. The only problem is that it requires going through a two minute process to change the IP address the domain points to in the control panel and we´ve got about 4,000 domains! I think I might need a sleeping bag and a lot of coffee that night!

All the problems are solved! After spending all day pouring through javascript, Googling the forums to death and non stop testing I´ve finally cracked it! Some users were having trouble entering data on our site and I finally managed to figure out that most of them were using IE5.2 on the Mac. Turns out that the features of the Rich Text Editor I was using along with similar editors is powered by the browser. I kind of knew this but didn´t know how it was done.
 
Turns out that its implemented in IE in versions 5 and above (but not on the Mac) using something called MSHTML and in the various Mozilla browsers through Midas (I think). These are pretty much compatible but if the browser doesn´t support one of them then your users have no way of using one of these editors.
 
I´ve just had to go through and change the sites so that those who haven´t got up to date browsers can stick text in using standard textareas. Just to make it a bit neater I´ve hidden them inside hidden divs until they click on a link to show them. Not too bad considering.

… something gets blown up! Last year I went to France camping for two weeks. We managed to get through the first weekend before I got a phone call saying the building had lost power during a thunder storm and the internet connection had gone down. Back then all internet traffic was routed through the main server and this had lost power and needed to be booted again. Since I´m the only technical person in the company and the Linux box needed the etho interface bringing up so thats why I got a call. After 5 minutes of my boss not being able to log in to the box because the keyboard he was tapping away at wasn´t plugged into anything, it was up again and I was off to the beach.
 
This week I had another few days off and the last night that I was off we had another thunder storm. The lights flickered a bit in the pub but apart from that I didn´t think that anything would be wrong. Especially now that we have seperate routers, UPSes on the main servers and most things were set to automatically boot up again on a power failure. I got to work though and it turned out we had got a few more lightning problems but all that had happened was a few fuses in the power sockets had blown. They managed to sort this out by the time I got there so I just had to check what had been going on with the automatic invoicing system and get it to run off the ones that were lost when a print server was down.
 
Next time I want some time off we´re getting a 3 day forcast first.

My server requirements are fairly simplistic compared to some setups as I only use Apache, PHP and MySQL on my actual web servers. All the other stuff like DNS and email are handled by other servers managed by our hosting company. This means that if I keep a copy of the httpd.conf and cron job files then all I need to do is take regular copies of all the web files and the MySQL database store. That way I can get back up and running even if I get a complete hard drive failure within a few hours using the original server image. Sorted.
 
The only problem is that I want a nice automated set of scripts that will take daily backups of the database, weekly backups of the web files and do it in a completely redundant way.
 

After a fair bit of messing around I?ve managed to get two identicle Linux boxes that I?m going to use as development servers, one for each of the dedicated servers we?re getting. Using FTP from protected directories and mysqldump run by cron with plenty of error checking and reporting by email. The same thing is done by tar and gzip to the web files. All these files are then copied by a third server onto our windows file server. All these servers perform error checking on each other and if something hasn?t been done then they report it by email. The final stage is to FTP the weekly backup to an offsite backup server which also performs error checking on the main office servers. This way, if something is not working, it gets reported so everything from individual server crashes up to complete power failure or loss of internet connection on one of the sites is flagged up.
 
Just got to wait for a week or so till we get through the hassle of moving the servers over and then I can test this whole lot.