Created: 29 Oct 2004
I’m not quite sure what’s wrong with me at the moment; I just feel tired a lot of the time and last weekend I had a niggling cold that wouldn’t go away. I certainly haven’t been feeding myself very well or making sure I get enough sleep.
This should partly explain a really stupid mistake I made at work
today. I was logged onto one of our Foundry ServerIron load balancers,
with the intention of deleting a virtual server called catering
[*]. Now Foundry kit has tab completion on commands, which is a good
thing. I’ll type something like show server \[Tab\] and get a list of
possible commands. It does not, however tab-complete user defined
names, such as those of virtual servers.
The astute, technically able reader will now have a pretty good idea what happened next:
#no server virtual cat\[Tab\]
Server cat found, subsequent tokens ignored.
So, that was the cat virtual server gone. The tab completion
algorithm decided that my command had the correct syntax and it could
go ahead and submit it. Completely accidental removal of a website in
a few seconds. The irony is that in my favourite shell, bash, I use
tab completion as a safety mechanism. If I’m typing a potentially
dangerous command, such as removing a whole directory and all it’s
contents, I’ll hit the tab command to check I have the right
directory. No harm will ever come of using the tab key, as it will
never submit a command. This is a hard habit to break.
Meanwhile, back at the company with the suddenly missing website, the sys admin needs clean underwear and a plan. I have a five second evaluation of what I can do to fix this. An idea; I’ll recreate the virtual server! I simply look in the backup of the load balancer config and paste in the relevant lines. It’s at this point that I discover that the deleted virtual server has not, in fact been deleted. The ServerIron is waiting for all the connections to close before it deletes it. So I can’t recreate the virtual server, as it still exists. At this point I very quickly move the load balancing of this virtual server onto our other ServerIron and the site’s back in under a minute.
I’ve moaned about the perverse aspects of the ServerIron configuration interface before. It’s unlikely that their lead software engineers are going to read the bloggings of an obscure sys admin and incorporate his suggestions into the next release, so I’ll have to do something to stop this happening myself. Less complacency, more caution, an acute awareness of which interface is being used and, most importantly, more sleep.
[*] Actually, it’s not called that; names have been changed to protect the guilty (me).