When doctors need to make life and death decisions with images from state-of-the-art imaging equipment, network engineers know that good planning and redundancy can be essential. Doctors and nurses at San Antonio Community Hospital (SACH) must have access to as much information as possible to diagnose their patients' ailments whenever needed.
SACH is located in Upland, Calif., some 35 miles east of Los Angeles. The region represents one of the fastest-growing metropolitan areas in the United States.
Fulfilling its commitment to provide leading-edge medical care, SACH upgraded its radiology services with a picture archiving and communications system (PACS) three years ago. Because PACS images can be created, stored and shared anywhere, healthcare providers have come to value the immediate access to the diagnostic information. For that reason and others, a PACS network failure is unacceptable. The loss of radiology images would delay critical diagnoses. So, SAHC set out to create a PACS network with reliability equal to a telephone switch and with recovery time of less than 100 milliseconds.
In order to replace x-ray film, SACH had two major requirements before installing the PACS. If doctors were going to depend on the digital images to make life-and-death decisions, the network that delivers the images must always be operating. The previous network infrastructure could not have handled the increase in network traffic, so the facility would have to install and manage a new network that never fails.
When completed, the SACH PACS network would need to process 11,000 exams per month from 30 image capture units of various types and present the images to 25 color and black-and-white diagnostic workstations. The images would be archived up to three terabytes on RAID5 disc storage and high-speed tape jukebox for historical data. Doctors would need to be able to access a PACS image in two seconds on-demand.
Currently, SACH stores and displays approximately 30,000 files per month from modalities that include, ultrasound, CT, MRI, nuclear medicine, computed radiography, digital radiography and fluoroscopy. Both the network and the wireless system must scale for bandwidth for the heavy traffic load.
The most important requirement is that the network must never fail. It must be operating 99.999 percent of the time. That prerequisite equals less than five minutes of cumulative downtime over a 12-month period. If the network were to fail, it must recover so quickly that the computers on both sides of the network would not know the network had failed and there would be no interruption of services.
The possibility of failure using the network for PACS was clearly illustrated to the IT staff after a visit to a nearby hospital that recently installed a new PACS. In the first three months of operation, the PACS network suffered two so-called "broadcast storms." Each time the PACS network was down for six hours, leaving the hospital staff without radiology images for diagnosis. The hospital had been built to withstand an 8.5 scale earthquake, but a data network broadcast storm could bring down the entire PACS and leave doctors without access to diagnostic information. This PACS network was based on Layer 2 switches.
A broadcast storm occurs when a faulty computer talks too much. The bad computer transmits particles or packet fragments. Fragments are not legal packets; they have no source or destination address. Since there is no destination, the would-be destination never acknowledges receipt of the message. Without that acknowledgement, the source simply continues its attempts to transmit again, sending more fragments, filling the network with particles and generating a dust storm in the local network segment. This is called a jabbering node. Other computers become quiet, waiting for a break in communications before they can talk.
In a broadcast storm, the network can grind to a halt when the system is flooded with incomplete messages from damaged or intermittent computer network interfaces. The Ethernet media access uses carrier sense multiple access protocol (CSMA). Multiple computers access shared media, and each computer listens to see if other computers are talking (also known as carrier sense). If another computer is talking, the first computer listens passively. If no other computer is talking, the computer transmits its message.
Layer 2 networks use forwarding bridges. A bridge sees