Memorial Hermann Health System
Memorial Hermann Health System is the largest not-for-profit health system in Texas, with 12 hospitals — including one of the nation’s busiest Level 1 trauma centers — and 200 doctors’ offices, clinics, and professional buildings spread across the greater Houston metroplex. Both patient care and record keeping are highly dependent on having reliable and secure wireless network access. With facilities that can be 50-75 miles apart and a team of just 3 engineers, Memorial Hermann Health System needed a solution for remotely diagnosing and troubleshooting wireless network problems that would reduce the need for travel and shorten the time from problem identification to resolution.
NETSCOUT’ AirMagnet Enterprise with SmartEdge Sensors
NETSCOUT’ AirMagnet Enterprise allowed Memorial Hermann Health System network engineers to reduce their travel time between facilities by 70 percent, and overall troubleshooting time is down by 50 percent. Memorial Hermann Health System now has deeper insight into the devices that are on its wireless network, with proactive alerts for suspect or rogue devices.
Memorial Hermann Health System is the largest not-for-profit health system in Texas, with 12 hospitals — including one of the nation’s busiest Level 1 trauma centers — and 200 doctors’ offices, clinics, and professional buildings spread across the greater Houston metroplex. A few years ago, recognizing a growing need for data networking capacity, Memorial Hermann Health System began to update its network infrastructure. 1 Gbps connections within the data center and between hospitals were replaced with 40 Gbps connections within the data centers and 10 Gbps connections out to major facilities. At the same time, the organization began updating its Wi-Fi network to 802.11n.
The wireless network is made up of more than 5,000 access points (APs), serving both Memorial Hermann's medical and administrative staff, as well as guests. At any given time, there are roughly 16,000 wireless devices and 9,000 guests on the network. In addition, an increasing number of medical devices and diagnostic equipment have built-in wireless networking capabilities. Many send data directly to the healthcare provider's electronic medical records (EMR) application.
In fact, Memorial Hermann Health System’s EMR system is heavily dependent on wireless networking. Wireless-enabled computers-on-wheels (COWs) and in-room scanners allow doctors and nurses to deliver positive patient care at the bedside using applications like PathNet®, RadNet® and CareNet® from Cerner Corporation with records syncing wirelessly to the central EMR system. Given the importance of wireless access to patient care and recordkeeping, Memorial Hermann Health System needed a solution that would help to quickly resolve wireless connection problems and maximize the efforts of its small wireless engineering team.
“I’ve used AirMagnet products for about 6 years now. They have proved to be an exceptional solution when it comes to spectrum analysis and wireless intrusion detection and prevention.”
– Ty Hall, Data Center Infrastructure Manager, Memorial Hermann Health System
With such a large number of facilities spread out across a dense urban and suburban metropolis and a team of just three wireless engineers, one of the biggest issues facing Memorial Hermann Health System was simply logistics.
“If a nurse placed a trouble ticket about a COW dropping off of the WLAN, it could mean a drive of 50-75 miles through some fairly heavy traffic for one of our three engineers,” said Ty Hall, Memorial Hermann Health System’s Data Center Infrastructure Manager (and previously its wireless and voice network architect). “It could take hours just to get somebody there, then they’d have to try and figure out what was going on.”
Memorial Hermann Health System also has a very complex WLAN environment with many devices competing for spectrum. In addition to providing wireless access to computing devices like COWs, laptops, tablets, etc., Hall and his team needed to account for medical equipment like radiology machines that create their own ad-hoc wireless networks. With so many frequency hopping devices and other equipment that creates co-channel interference, Memorial Hermann Health System eventually had to segregate its network into a 5Ghz band for their own production devices and 2.4Ghz band for medical equipment and guest access.
“Pumps, X-ray machines, radiology equipment… just about every device in the hospital environment is going wireless these days,” notes Hall. “But usually when one of these devices creates its own ad-hoc network, it doesn’t consider the impact of its own power or channels, so we have to figure out how to deal with it.”
In that kind of an environment, the need for reliable diagnostic information is critical to quickly resolving problems. But, the built-in spectrum analysis capabilities of Memorial Hermann Health System’s wireless APs were not sufficient. And, vendor support was less than optimal. “I just didn’t trust our wireless AP vendor to give me correct information about the status of the APs on our network,” said Hall.
Finally, since most — if not all — of the data across its wireless network is highly confidential and subject to the provisions of HIPAA, Memorial Hermann Health System needed to take proactive measures to protect it from unauthorized access. That goal is complicated by both Memorial Hermann Health System’s distributed environment and human nature. “It’s not uncommon for doctors to bring their own APs into their offices,” said Hall.
“We’re really pushing wireless because it make sense from a cost and management standpoint,” said Hall. “The more users go wireless the fewer switches we’ll need. The facilities we’re going to build in the future will have 80 percent fewer switches and more access points. But that increases the need for wireless security. As a healthcare provider under HIPAA, we really need to know what’s happening in our own environment.”Solution
Memorial Hermann Health System chose NETSCOUT’ AirMagnet Enterprise to provide spectrum analysis and wireless intrusion detection/intrusion prevention (WIDS/WIPS) capabilities. Even though the APs Memorial Hermann Health System uses provide some native spectrum analysis functionality, Ty Hall counts on an overlay of AirMagnet Enterprise and the associated SmartEdge sensors to provide more comprehensive insight into the behavior of devices on the network, and powerful alerts for specific devices and activity.
“I’ve used AirMagnet products for about 6 years now,” said Hall. “They have proved to be an exceptional solution when it comes to spectrum analysis and intrusion detection and prevention.” Hall noted that AirMagnet Enterprise provides a wider range of signatures on which to build alerts and provides deeper spectrum analysis than competing products.
With AirMagnet Enterprise, Memorial Hermann Health System’s wireless network team now has the ability to remotely diagnose connectivity problems. Now, when a user complains about a connectivity issue, instead of an engineer getting into a car to diagnose the problem, AirMagnet Enterprise allows the engineer to see whether the user is connected to the network, whether there is any nearby interference and identify a potential culprit.
“We’ve reduced our travel time by about 70 percent,” said Hall. “That along with the added insight from AirMagnet Enterprise has helped us see an overall reduction of about 50 percent in the time it takes to troubleshoot a problem.”
Hall applauded AirMagnet Enterprise for the intuitiveness of its interface, and appreciates the fact that his engineers can use the product with very little training. It’s also very easy for Memorial Hermann Health System engineers to create reports from the application.
The security features of AirMagnet enterprise are crucial in the regulated hospital environment. Memorial Hermann Health System can now block malicious users by the MAC addresses of their devices. “We’ve set up proactive alerts that notify us when rogue devices or devices of a certain signal strength enter the network.”
And that’s not all. “We’ve even been contacted by the police to keep an eye out for certain stolen devices that may show up in our environment. We’ve been able to set up alerts for specific MAC addresses that have allowed us to help the police recover stolen property. AirMagnet Enterprise really is the market’s top WIPS/WIDS solution.”