Friday, 9 September 2016

Story Time : Fantastic Rack Mount Mistakes #3

Technically this is not a rack mounted mistake, but it was a mistake made by someone with a rack mounted unit.  A loooooooottttttttt of years ago, around the turn of the millennium in fact, I worked at a company where they used a Windows 2000 Server to collect a series of information, from memory, I think it ran Websphere and Jakarta Tomcat as middleware, connecting to a Microsoft SQL Server back end.

This server gave me nothing but trouble, I had to bring it down to publish new software to it, then bring it all back up again, because of some wierdness, and worst of all, I was only a user on the machine, I never saw it, was never in the same room as it, it was hosted in a data centre far away from me.

Remoting into it on late 1990's internet (because they'd not upgraded) was painful at 14kpbs, so much so that connecting to it from home was easier, where I had the first generation "Diamond Cable" (later NTL, later Virgin) broadband... at a blistering 512Kbps.

The main problem was, I needed someway to get the server to tell me it's IP, it could change, and the network guy at work used to handle this by crystal ball gazing, and copying the new IP into an e-mail and sending it to everyone.  Of course at home, I didn't get my work mail... Humpf.

The system however was all just Java, and after about 18 months I got the chance to move up the suckling chain of management within the project, and I immediately pointed out to the directors that we could drop all this Windows stuff, and the expensive licensing, by going to this new system called "Linux".

As a result I ended up at an open day presentation in the middle of Birmingham, with speakers from RedHat and Suse.  The talks were pointless, the guy from RedHat wore a T-shirt saying "Suse is shite", and the guy from Suse wore a suit and talked about professionalism; not very constructive, it all wanted to pull the admins in the room into the "are you a pro, or are you a hacker" division.  Neither talked about the distribution.

I therefore decided to learn about Linux myself, and I started a new server, initially just an old Pentium II machine with 32Mb of RAM, it was soon found to be a little bit better and a hell of a lot cheaper than the Windows Server, so I got a new shiny machine (my first commercial rack mounted server, I forget the model, I'm pretty sure it was a Compaq, as I'd seen the Proliant at my previous place of work and think I told the boss to get one).

This machine arrived, and it ran seamlessly pretty much from the get go, as had a couple of disk deaths in it's time, but there was no problem with it.

With the problem of it being in the remote location however, I had a trick, I created a script, like this:

ifconfig > config.txt
traceroute >> config.txt
cat config.txt | mail -s BootMail
rm config.txt

The result?... I had it send me a mail whenever it reset, so I got the IP information, and a trace route listing to the outside world (I think the original place I used to trace route to was Altavista!  And it didn't go to hotmail, it went to my old NTL provided e-mail address.

What's the mistake here?... Well, that NTL e-mail address linked to my virgin e-mail address, which forwarded to my MSN address, which turned into Hotmail.

And... You may have guessed it... The other day, that very server was turned on somewhere, somehow... It had a network connection, was powered on, and it sent me a mail!

It was in Worcestershire, I think, and I could still SSH into it!

I changed the daily greeting to tell them they're server is insecure, disabled the script which was e-mailing me, and this pretty much ends our story.

The failure?  Don't plug things into your network, into your rack, unless you know what they're trying and going to do.  EVER!

No comments:

Post a Comment