8

I am writing a novel in which a researcher attempts to work against a study he is a part of by purposely sabotaging operations on the research compound. I am wondering if the following is plausible.

"Bob" starts a small fire in the server farm containing the primary servers for data collection in the study. He has "Steve" (unaware of the sabotage) in position to put out the fire so that Bob can perform an emergency back-up of the data to the study's secondary off-site servers in case the fire spreads (the data would otherwise have been backed up at 1 a.m. like any other day). Bob does this because at this point in time he is hoping to make himself look like a hero as to not draw any suspicion for his actions later.

Does that make sense? Would a fire pose a reasonable threat in this environment? Does the premise of needing to perform an emergency data store accurately reflect something that could occur in real life?

I have a little leeway in plausibility as this is set some time in the future, but want it to still be reasonable to readers who are more informed about these matters than me.

If anyone has any suggestions for other possible (but easily resolved) physical threats, I would greatly appreciate those as well.

Thanks for your help!

  • 14
    In a fire, you evacuate. You don't start fiddling with backups. – schroeder Dec 16 '14 at 19:49
  • 2
    Not my realm of expertise, but I would lean more towards something with the HVAC - broken AC units, for example, will quickly lead to massive server overheating and crashes, sometimes worse. You could try to make that extreme... In that scenario (as one of the answers stated a threat to machines but not people), it would be more reasonable to deal with up-to-minute backups, since the protocol is (usually) to shut down servers in an orderly fashion and perform failover to the backup site - but not in an immediate "flip the switch" fire style. But then again as I said, this is not my expertise. – AviD Dec 16 '14 at 22:05
  • 2
    A search warrant – JMK Dec 16 '14 at 22:24
  • Instead of asking if how is acceptable, it would be more useful to understand what you want the characters to do and why. Once we know what you want to do, it would be easier to go into how that would actually occur. – Paraplastic2 Feb 3 '15 at 13:55
14

Most things of importance placed in a data center have a sister installation in a physically separate data center, and if there was a reason to "fail over" from the live to the backup site, it's completely plausible that things like data shipping will occur as part of it. It also will result in a flurry of manual and semi-automatic administrative activity, which could plausibly allow someone to exploit a toehold in the environment and expand it.

It's a standard trope that response plans and disaster recovery plans can be exploited as they cause predictable actions on the part of the victim. Consider Star Trek Into Darkness, where the villain bombs an installation for the sole purpose of triggering an executive meeting. The children who pulled a fire alarm in order to flush their classmates out of the school to be shot at (Tennessee? Georgia? Arkansas (see Michael A's comment below) A dozen years ago or so). Appleseed Ex Machina where the villian replaces the programming on the backup computer, then orchestrates an event which will cause failover to that computer.

If you can find interesting incident response plans, read them and see what predictable behaviors can be used by an attacker.

  • Thanks for your insight (and also the link to the TV Tropes site--I discovered it a while back myself and found it very useful as well). I appreciate your thorough response and will keep the information in consideration going forward. – An Author with a Question Dec 16 '14 at 21:01
  • 2
    The shooting you're thinking of is the Westside Middle School massacre. – Michael A Dec 17 '14 at 0:24
5

A fire is a real threat in a datacenter, but it would certainly not be handled with an emergency data backup. Server Fault has some discussion about what to do in a fire; the consensus there was that as soon as it was identified as an actual fire, you press the emergency poweroff button and release the fire suppressant on your way out the door, and call the fire department to come put out the fire. You aren't bothering with an emergency backup because you're evacuating the building and letting firefighters do their job; you also shouldn't need one because backups are done routinely so that you don't have to do them in a hurry when things are already failing. Even if the idea is that Bob's being a hero by taking the risk, the fire department cuts all power to the building as one of the first things they do when they get there.

Other situations might involve rapid transfer of data from main to backup datacenter, but a fire is not one of those.

  • 1
    Thanks for your response. Could you provide me with an example situation that might involve the transfer of data from the main to a backup datacenter? Another commenter mentioned acid corroding the server stack, but do you have any other suggestions? Thanks again! – An Author with a Question Dec 16 '14 at 20:57
  • 2
    How about a power outage? Many places have a limited battery backup and/or a generator (may involve some marginally-heroic hauling of heavy cans of diesel) that would give them a small amount of time to do things like backups. Other than that the usual reason I've had to do something like this is gradual hardware failure (e.g. a raid controller showing more and more checksum failures), but that's rarely very urgent. – lmm Dec 16 '14 at 23:12
  • 1
    How many firefighters actually know how to deal with a fire in a data center? I know of one case where firefighters arrived at a fire in a building housing the electronics used for traffic control on a railway. Since those firefighters didn't know what safety precautions were necessary due to the electric installation, they just watched the building burn to the ground. – kasperd Dec 17 '14 at 4:44
  • 1
    @kasperd Most can, I think. If you're worried, you can work out fire response plans with the fire department beforehand. The relevant point is that if you aren't the fire department, you can't deal with a fire safely. That said, this is why many datacenters have an EPO: it turns an electrical fire into an ordinary one. – cpast Dec 17 '14 at 14:06
  • 1
    @cpast It's more complicated than that. If you want to ensure there isn't any power anywhere in the building, you would have to cut power, before it enters the building. But usually that isn't enough either due to any UPS present. Moreover the fire department might still not know what the procedures for cutting the power is, and if they have been executed. In the case I mentioned before, the risk wasn't any different from putting out a fire in an ordinary home. The reaction from the fire department was entirely due to lack of knowledge. – kasperd Dec 17 '14 at 14:32
2

If you want a physical threat, you'd need something that would be a threat to data, but not people. Acid corroding the server stack, for example, might be better. Bob could trigger a backup in safety while racing against time to save what data he could before the server was irrecoverable.

  • 1
    Acid probably doesn't exist naturally in a data center environment though, so it'll be a clear indication of sabotage. Fire, OTOH, can plausibly self ignite in a date center. – Lie Ryan Dec 17 '14 at 3:16
  • @LieRyan I agree, but given that it takes place in the future, it's the idea of acid that is my point. – schroeder Dec 17 '14 at 18:11
1

Keeping in mind that this is fiction, your scenario could work given some qualifications.

Perhaps there was some major breakthrough that day in the study, so it was worth the risk to backup the data on demand.

  • Thank you. A major breakthrough is part of what is at play insofar as the narrative is concerned. Knowing that it could work given a few qualifications is helpful, as I do have some in place that are internally consistent with the rest of the manuscript. – An Author with a Question Dec 16 '14 at 20:58
1

There are a few more things you need to consider which may or may not effect what you desire to write. Firstly being is that real datacenters with SSAE16 certifications have in place controls and monitors which give a full accounting of all the goings on.

Authentication and Authorization. When I go into my Tier 4 datacenter, I am required to use my biometric scan and keycard no less than three times before I am even on the server floor. This means, that anytime I am in the facility, My presence and location is exactly known.

Secondly, every single room, and every single angle is monitored with video. You can't hide in a datacenter that is taking their security seriously. Additionally, an armed guard is on duty, and many check and scan me and prevent certain hazzardous materials from entering into the datacenter. There is a long laundry list of items I am allowed to take into the datacenter, and it is religiously checked.

Lastly, Datacenters have automated fire response systems. In the old days Halon was used, but now we see chemicals like FM200 (there are others) used which are designed to put out the fire without hurting the equipment housed.

As was mentioned before, once a fire alarm hits, SOP is for people to evacuate, and let the rest of the security systems do their job. These systems are normally audited on a yearly basis to ensure their functionality.

Depending on the importance of the data, it may be required to be highly available. In addition to backups, there are situations where replication technology is used. Whenever you use a service like Amazon or Google, any other major retailer, they often have multiple datacenters running which keep a running copy of your data in OTHER datacenters.

So onto your question...

Physical threats need to address all these factors before they can be an actual threat. TV and movies almost always get it wrong. Star Trek is among the worst for completely and blately just out right failing when it comes to security, physical or otherwise. Their writers are absolutely clueless. If you need to stress the need to backup the data, than the datacenter/server farm would likely not be very well designed in the first place, or the owners simply couldn't afford to provide adequate controls and monitors, both which do happen in real life. I've been in some very ill designed 'datacenters' that barely qualify to own the name.

There is much much more to it than this, but this is just the tip of how the big boys operate in the real world, so your situation may differ, but hopefully this gives insight and context for your novel on what goes on in a datacenter.

Source: CISSP

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.