In-Depth
Case Study: Securing Systems You Don't Directly Control
The University of Colorado Hospital struggled to keep its network secure to comply with HIPAA regulations, made difficult because IT didn't have direct control over some connected systems.
How do you secure a hospital network when you don’t control all of the machines touching it?
Joe Bajek, director of IT for University of Colorado Hospital (UCH), faced that dilemma as he struggled to keep his network secure and auditors happy in the wake of the Health Insurance Portability and Accountability Act (HIPAA).
UCH, a non-profit, teaching, and research hospital partially funded by the state providing community medical care, is in a somewhat unique situation. Its physicians don’t work for the hospital—they work for the university. For ease of management and cost savings, different entities—UCH, the university, or the separate billing entity—control different buildings, even when they’re multi-function. For example, doctors at a university building, running computers supported by the university, often need access to the hospital network, yet the hospital doesn’t support (and therefore has no control over) what’s on the doctor’s machines. In short, it can’t guarantee anti-virus programs or application patches are up to date.
Machines like these don't mesh well with HIPAA regulations. “We get audited both by the hospital’s internal audit and externally on a regular basis, which is part of our nonprofit status. We need to be as squeaky clean as we can be,” says Bajek.
Luckily, HIPAA helped produce an answer. A few years ago, after an audit by StillSecure—then known as Latis Networks—UCH scored relatively high vis-à-vis other healthcare providers. Still, the audit produced a recommendation: consider intrusion detection. Bajek had been trying, unsuccessfully, to fund penetration testing in lieu of more high-level information security staff, for which he didn’t have funding. Through negotiations, however, he agreed to be a beta user of StillSecure’s Border Guard, an intrusion detection product.
Bajek says he was initially comfortable going with Border Guard, “which was SNORT-based under the covers.” Yet as with any intrusion detection/prevention product, it takes time to tune. “We’ve spent the last year getting our feet wet with it.” He’s also been able to add a two-person security team to his IT staff of 80.
UCH monitors all traffic with StillSecure Borderguard, using two nodes—one for inbound, the other for outbound. “Most networks that are connected to the Internet are not good neighbors,” he says, owing especially to mis-configured PCs. He watches his carefully. “The last thing I want to be is not a good neighbor.”
HIPAA is less about requiring specific technologies and more about creating a security-focused organization. Likewise, adapting intrusion detection required a cultural shift at UCH. “If you don’t have someone looking at it, then it doesn’t matter what it’s saying,” says Bajek. The two-person team checks the Border Guard console at least two or three times per day for alerts and to fine-tune it. “We’re really tuning that to a point where later this year we expect to turn on paging,” so staff can trust the accuracy of alarms and devote more time to other tasks.
It won’t ever be fully automated. "Applications will vary. There could be zero-day exploits,” Bajek notes, “but an awareness of what your network looks like is probably the best tool for a security team so you can understand, baseline … and understand whether something is a threat or not.”
When it comes to using intrusion detection, Bajek cautions, “You certainly spend a lot of time at the head end tuning it to understand what it’s telling you.” That’s mandatory, he says. “If you don’t spend a lot of time putting in protocol analyzers and learning what your traffic is like—and most of us don’t—then you spend a lot of time in reactive mode.” More time up front means greater familiarity with exactly what’s going on with the network, plus the benefit of a more automated alarm system. “I have a million other tasks for my security team to deal with, and staring at an intrusion detection console is not on my list. So the more automated it can be, the better.”
Beyond Intrusion Detection
Beyond finding intrusions, however, Bajek wanted to monitor for vulnerable machines inside the network, which consists of three core nodes on two campuses connected by multi-gigabyte Ethernet channels. There are 3,000 desktops and 150 servers, plus medical equipment running Unix or Microsoft operating systems, for which vendors must release specialized patches, something they often delay.
UCH IT ensures hospital PCs are locked down, yet not everyone accessing records—such as physicians—is necessarily using a hospital PC. Since physicians work for the university, university IT supports their machines. Also, Bajek doesn’t have the time or staff to manage every machine touching his network.
One difficulty with monitoring today’s networks is that they’re switched. “In a switched environment, versus [the] old days with a hub, you won’t know—unless you’re looking at the stream between workstations—who’s infected.”
He found a product to watch the streams—also from StillSecure. Called VAM, the software scans all computers touching the network without needing an agent running on the computer.
First, however, UCH grouped any machine—hospital, physician, contractor—accessing the network into a virtual LAN. Now they get scanned with StillSecure VAM. If a machine is doing something questionable, “the first reaction is containment,” says Bajek. “Our first responsibility is to our patient data, and trying to protect that.” Luckily, he says, “most of the stuff we’re seeing is e-mail based, so it’s usually pretty easy for us to determine who clicked on the e-mail that started spewing the virus. It’s tough, because you can’t sit there and ratchet down e-mail, because it’s a phenomenal communication tool.”
If VAM finds a computer with problems, and UCH IT doesn’t support the machine, UCH IT can contact university IT to, say, upgrade the operating system patch level on a doctor’s computer. “Most of the time the physician doesn’t even know it’s been done. Our customer base shouldn’t be worried about things like that,” he says; they need to focus on patient care.
To ease reporting and administration, UCH also runs a VAM-distributed scanner at the satellite campus, which pipes information to the central VAM server via SSL. A server version of VAM also watches medical equipment with built-in operating systems, so even if equipment manufacturers don’t release patches in a timely matter, UCH IT can proactively protect the machines against known vulnerabilities.
If he wasn’t using VAM, Bajek says he’d have to find “an IP port scanner, some command line product, then interpret what it’s telling us.”
Though he’s got intrusion detection and vulnerability monitoring, Bajek also focuses heavily on educating users. “Technology is about 10 percent security, risk assessment and threat management, and 90 percent is cultural. If you can teach people not to read that e-mail and to call before they open it, you’re winning the battle.”
====
Related Stories:
Top Three Security Problems Remain Despite Increased SpendingCompanies turn to managed Service providers for help; desktops dominate security budgets as patches average $234 per machine. http://info.101com.com/default.asp?id=5845
Best Practices in Security Training Worms and legislation dictate the need for security-savvy employees; here's how to train them and reinforce human nature.http://info.101com.com/default.asp?id=5846
Companies Nervous as First HIPAA Deadline ApproachesAutomated monitoring key to HIPAA compliance; Symantec fields questions from worried users http://info.101com.com/default.asp?id=5847
About the Author
Mathew Schwartz is a Contributing Editor for Enterprise Systems and is its Security Strategies column, as well as being a long-time contributor to the company's print publications. Mr. Schwartz is also a security and technology freelance writer.