Halloween is upon us, and everyone loves a good scary story. To get you ready for Trick-or-Treating, and to provide a break from the non-stop monster movies on TV this week, we’re going to share 21 of our favorite SysAdmin horror vignettes. Take a read through them all, see just how many will resonate with you in a cringe-worthy way, and then share your own personal horror story in a comment below. We can’t wait to read what you can come up with, especially since we all know the truth is at least as scary as fiction!
1. An admin ghost story
It wasn’t that the former employee’s admin account was still enabled. It wasn’t even that the password was set to never expire, or that password last set was years ago. The most terrifying thing about this admin account was that, even though the user was fired months ago, last login time was ONLY TWO DAYS AGO!
2. He’s inside the network
The IPS started to alert all the admins that something was wrong. An attempt to break into the VPN server was in progress, with someone trying to brute force credentials. The source address of the attack seemed familiar. In fact, it was the NAT address for one of the main offices. The hack was coming. FROM. INSIDE. THE. NETWORK!
3. Please let it work please let it work please let it work
Nothing is scarier than that moment when, just after making a change to your email system that wasn’t supposed to hurt anything at all, and you send that test email to your own corporate account from your personal account, when it doesn’t arrive. Well, nothing scarier than the second moment when it still hasn’t arrived. Then there’s the third moment when it still hasn’t hit your inbox.
4. You’ve got backups, right?
When the junior sysadmin saw duplicate data on the X:, Y:, and Z: drives, he thought he would free up some space on the network by deleting the redundant data. Unfortunately, he didn’t realize that the system mapped all three drive letters to the same network location. Suddenly there was A LOT of free space on the server. Too bad nobody could use it!
5. Know anyone at the NSA?
Encryption is a great way to prevent unauthorized access to data, but when you’ve encrypted that data and then cannot remember the passphrase to unlock the key to decrypt it, you are in a world of hurt. As bad as it might be to write down a passphrase, it’s not as a bad as being unable to decrypt the only copy of 320GB worth of data.
6. They’re here!
Nothing is much more fun than when HR “forgets” to submit the new account provisioning forms for the army of people who are starting today, in the lobby now, and won’t really need their laptops or to log on until probably 10:30, maybe even 11:00 this morning. That’s time for you get them setup, right?
7. You know you have to update that stuff, right?
Like a lab experiment gone horribly wrong, I once worked to clean malware from a machine that a user brought in from home. We stopped counting when we hit 100 unique types of malware on the machine and finally had to give up and just wipe the drive. The user couldn’t understand why their antivirus software didn’t protect them. After all, it was Windows 95 and the antivirus definitions were dated 1995. This was late in 2003!
8. You emailed how many people how large a file?
A user decided to create a very pretty ad slick and saved it to PDF, and then sent it out to an 11,000 user distribution list. Not being familiar with PDFs, the user saved it for printing, not the web. I’m still not sure what 11,000 * 14 MB comes out to, but it equals a dead Exchange server, a bunch of entries on blacklists, and so many NDRs that they killed the inbound system too. We still call it “The Perfect (Mail) Storm.”
9. It’s dead, Jim.
If you really want to experience pain and suffering, get up early, drive to the airport, fly to the customer, wrangle your way through traffic driving a rental car, find your way to the customer, get to the conference room to make your big presentation to the customer, connect your laptop to the projector, and nothing. No boot, no backup, no nothing. Somewhere along the way between home and the client, the laptop died. Take it from me…always have a backup plan. Dropbox, SkyDrive, email yourself a copy, a USB key… something to fall back to in case the worst happens.
10. Give me what I want, and I’ll go away.
There’s a reason I am so quick to go to a user’s office or cube instead of having them come to mine. When I’m in their cube, I can leave at any time. You’ve probably all run into the situation where a user comes to see you, and then you’re trapped in your cube because they just won’t go away!
11. With one key press, he killed 200 virtual machines.
Working with a SAN admin one time to “optimize” the storage array, I said stop just about .25 seconds too late, as he applied a new config to the production SAN and not the new SAN, dropping the disks for all the VMs in the datacenter. As it turns out, Nagios looks really cool when it’s all red!
12. Braaiiinnsss!
Zombies are scary. There are fast-moving zombies that scare the hell out of me, slow-moving zombies that just creep me out, and computer zombies that like to take down websites and spew spam. If you want to get really creeped out, imagine getting on site to find that an entire office’s computers were infected with malware and were all part of a botnet.
13. We’re trapped in a time loop and there’s no way out!
Doesn’t it always seem to happen that someone works a problem all day long, cannot figure it out, then calls you and everyone else into an emergency phone bridge to start troubleshooting at about 15 minutes before quitting time. And then when the problem is solved, and someone reminds everyone not to wait until the end of the day to call for help, and then the next time someone works a problem all day long, cannot figure it out, then calls you and everyone else into an emergency phone bridge to start troubleshooting at about fifteen minutes before quitting time. And then the problem is solved, and someone reminds everyone not to wait until the end of the day to call for help, and then…
14. Let me out!
It’s the start of a great weekend, and you’re completely stoked to have everything wrapped up so you can slip out early. You’ve got your stuff packed up, and you’re headed out the door when you run into your boss who just wants to chat for a few minutes to get a summary of how that project is coming along. By the time you’re done, you’ve missed that window and will be spending the next hour or so stuck in traffic.
15. There’s no such thing as too much redundancy
As a home office worker, Internet connectivity is pretty important. So you can imagine the abject terror that I experience working on a critical deadline when my Internet went down. Thinking I was prepared, I pull out my Mi-Fi device, fire it up, and only then discover that it didn’t work either. Being a Sunday night, there wasn’t even an option to go to the library or my favorite coffee shop. I had to poke around at each of my neighbor’s Wi-Fi networks until I found one that I could “borrow a cup of bandwidth from” to finish my work. As it turns out, since I had not used it in three months, my Mi-Fi had been deactivated. Now my cellphone has an Internet sharing plan too… just in case.
16. It’s alive!
Very early in my IT career, I worked in the technical support group for a retailer. A customer brought in a computer (what he called the CPU) that wasn’t working. There was an odd rattling noise coming from inside, and as I put it up on the bench, a bunch of small black granules fell out of it. My more experienced colleagues started to quickly back away, but I was too green to realize that what I was about to encounter when I opened the case was an infestation of roaches!
17. Epic fail
The wrath of the CEO when he could not find a critical email (that he had deleted, but no one dared tell him that) was temporarily assuaged when we assured him we could restore the message from a backup tape. Of course, that was nothing compared to the thermonuclear explosion that came about when we had to tell him that restoring the backups failed. Honest, he was a cross between Freddy Kruger and Hannibal Lecter for a week.
18. The magic smoke got out
Did you know that not all hard drives in servers are hot-swappable? Oh sure, that’s easy for you to say now, but when you pull that not hot-swappable hard drive out of the server and watch the magic smoke escape into the ether, you get to find out whether your backups worked or not.
19. It’s like it was out to get us. Just us.
I once worked for a company whose primary data center was in Orlando, and backup data center was in West Palm Beach. Hurricane Charley took out the power and connectivity into West Palm Beach first, and then made a beeline for Orlando, taking out that datacenter’s connectivity as well. The one good thing that came out of that opened another datacenter… in Chicago! The general consensus was that any storm that could take out both Florida and Illinois would likely be a big enough event that we wouldn’t care much about it.
20. The scary red glow like the embers of hell
If you’ve ever wondered what hell looked like, for an IT guy it looks like this. After another hurricane took out power to the datacenter, the batteries kept all the servers going, but the generator that powered the HVAC system wouldn’t start up. In the data center, the temperature quickly climbed to over 38 degrees C and all the bezels on the servers started to glow red. Since the main lights were off, the room was cast in a hellish red glow and the heat was just like you’d expect.
21. What’s using all the bandwidth?
We were trying to replicate some data to a remote office over a VPN tunnel but were getting really bad throughput. In trying to troubleshoot the issue, we determined that the office’s Internet connection was running at 100% utilization, which was strange since no one was in the office at that time. Looking a little deeper into the traffic, we found that a user had installed a peer to peer file sharing application, and had shared not only his entire hard disk but all the network drives he had mapped. We were being drained dry of every single file that the user could get too. Fortunately, there wasn’t any PII or other customer data in there, but all of our software keys were.
Did we scare you? Well, I hope not too bad, but I bet we shared some scary things that you’ve seen, and some others that you haven’t. Now, your turn. Tell us what the scariest thing you’ve ever seen in IT is. We can’t wait to read more!
Get your free 30-day 
trial
Get immediate results. Identify where you’re vulnerable with your first scan on your first day of a 30-day trial. Take the necessary steps to fix all issues.