Living Off the Grid

Planning for disaster recovery before you're stuck with a crippled cable service

The Internet was, as we all know, designed by researchers working under contract for the Department of Defense. The problem that DOD was interested in was survivability -- making a network that was robust enough to withstand the loss of a few major nodes, seeing graceful degradation of the rest of the network.

In other words, in the event of war, the Soviet Union could nuke San Francisco and we'd lose access to Berkeley and Stanford, but the rest of the network would still be fully connected. Of course, this is trivial if you have a fully connected network, but the goal was to have a sparse set of connections and a robust set of routing algorithms that could dynamically adjust for the loss of even major nodes.

So why revisit this bit of history now? Well, recent events have tested that very design, and in a way that allowed me to personally experience the pain of lost connectivity.

The event I'm referring to was the magnitude 6.7 earthquake that shook Taiwan the day after Christmas. Its epicenter was just off the Taiwanese coast and its force was such that most of the major trans-Pacific fiber optic cables were severed.

I happen to live in one of the least connected nations on the planet, Cambodia. Being an American geek, my VOIP phone and Gmail account are critical to my daily existence. At the moment most of my IP communications are operating at a snail's pace (even more than usual), if they work at all. The ability to reroute traffic through European links, and thus over the Atlantic cable systems, took a few days to get in place. Even now those links are minimally helpful, at least here.

Lessons learned? Well, I guess the good news is that I sometimes have access to my Gmail. As I understand things, the cables require special ships to repair them, and at least one of those ships was out of commission at the time of the quake, further delaying repair. So the cables are still out of service, but I have service, even if it is degraded. Score a point for fancy routing algorithms.

The bad news is the service is still degraded, a month later, and it isn't likely to get better any time soon. If this had been an attack or disaster at one of the major U.S. hubs (e.g. one of the MAEs), the whole fabric of the Internet would likely be crippled. Let's hope their recovery plans include having equipment available to implement the plan.

If I were still in charge of a corporate disaster recovery plan, I'd take this as a not-so-subtle reminder to review it regularly, and I'd make sure everyone understand the resources needed to implement the plan. It isn't the sexy part of security, but it is the sort of thing that can save your organization millions of dollars in the event of a major problem. That's probably worth few hours of your time every couple of months, don't you think?

Nathan Spande has implemented security in medical systems during the dotcom boom and bust, and suffered through federal government security implementations. Special to Dark Reading

Recommended Reading: