It's now clear that virtualized environments not only offer the opportunity to manage the data center more flexibly; they also offer a renegade administrator a more powerful avenue of attack. With virtualized environments, we can establish defenses in depth that far surpass what could be done in the physical world. But we are just getting used to this adaptable, shape-shifting world of virtual machines, and in some cases, we're creating greater exposures instead of mutually reinforcing protections in depth.
Take Shionogi, a North American subsidiary of a Japanese pharmaceutical firm. In July 2010, Jason Cornish, an IT staff member at Shionogi's operations in Atlanta, had a difference with his manager and resigned. A friend of 15 years at the company, who was not named in the court papers, advocated that he continue working for Shionogi as a contractor, due to his familiarity with its network, according to the case filed by the U.S. district attorney Paul Fishman in Newark, N.J.
Work channeled to Cornish stopped in September 2010, and later that month, Shionogi announced layoffs that affected Cornish's friend. On Oct. 1, the friend refused to turn over network passwords to the remaining Shionogi administrators, prompting his dismissal.
On Feb. 3, Cornish used a Shionogi user account, CVAULT, and a password accepted by the system to access a server where he had secretly installed a VMware vSphere client several weeks earlier. Shionogi operated a heavily virtualized infrastructure, and Cornish, working from a laptop that he had taken to a Wi-Fi-equipped McDonald's restaurant, proceeded to delete Shionogi's email, BlackBerry, order tracking, and financial management servers.
All in all, using the vSphere client to access vSphere's virtualization management console, Cornish with a single click systematically eliminated each virtual server on Shionogi's 15 virtualized hosts. While munching down the equivalent of a Big Mac and fries, Cornish eliminated the 88 virtual servers Shionogi depended on for its day-to-day business.
The fact that he was caught might lead you to think that Shionogi's defenses won out in the end, but it shows nothing of the sort.
His apprehension had more to do with the quick involvement of FBI Cyber Crimes teams, which existed in both Newark, where the attack took place, and Atlanta. The scene of the crime was the nearby Smyrna, Ga., McDonalds and the attack could be traced as coming from that site by tracing the attacker's IP address. Cornish was placed at the site a few minutes before the attack by his use of a credit card to make his $4.96 purchase. He must have been short of cash. Otherwise, his plan might have worked--and he might still be on the loose with no direct tie to $800,000 in damages to Shionogi.
It also helped that Shionogi discovered he had accessed its systems 20 times between the September layoffs and the Feb. 3 attack. They found the offending vSphere client and proceeded to build a case that lead to Cornish's Aug. 16 guilty plea. On Nov. 10, he will face a sentencing judge and be subject to up to 10 years in jail and a $250,000 fine.
But there's little comfort in justice being done in this case. Shionogi's procedures seem lax, and yet I know several instances where well-managed firms lost track of contractors who were periodically doing work for the company. Even in cases where former employees are swiftly expunged and contractors strictly monitored, every company struggles to protect itself against an inside job. The case against Cornish doesn't make clear where he obtained his working password. It's possible under the circumstances of this case that Shionogi took the correct action to protect itself from one disgruntled employee, then fell prey to another against whom no case could be made.
At a moment when IT staffs are being reduced, companies are particularly susceptible to inside jobs and much about this case smacks of an inside job.
Shionogi, however, might have followed the best practice of placing restrictions on IT administrator's privileges, restricting each to a set of defined servers. But Shionogi is not alone in assigning general privileges to trusted IT staff; doing otherwise sometimes means the people with the right skills can't access the right trouble spot. Shionogi might have set a software watchdog on who logged into which servers and who deleted servers, but many shops have no such protection in place capable of tracing a software event to a single individual.
What's truly interesting about the Shionogi case is not how quickly justice was done but how swiftly major damage was done--thanks to the management interface to the virtual environment. Shionogi was put out of business for several days until the virtual servers could be reconstructed and known, valid data reestablished.
I often get positive feedback on the amazing capabilities of IT managers in these emerging, virtualized data centers. But it would be wise to remember that with virtualization, it's not only the good guys who get "god-like" powers.
To see how VMware is extending its reach into data center operations, see VMware's Next Act: Operations Expert .