It was a series of unexpected events that resulted in this outage, during which two fatalities were reported.
Why Did Montgomery County’s 911 Outage Happen?
Ironically, the outage occurred when Montgomery County was in the process of upgrading their primary Public Safety Answering Point (PSAP) to a Next Generation 911 solution, and all of the county 911 staff was working out of their backup PSAP. The outage was traced to a malfunctioning heating, ventilation and air conditioning system that caused both the backup system and the main 911 communications server to go down.
While Montgomery County executives and residents are all shaking their heads in disbelief about the turn of events that caused the outage, this incident underscores the importance of properly planning upgrades and ensuring the multiple redundancies are in place for 911 centers.
Along with our partner Unify, MicroAutomation has implemented and maintained the 911 infrastructure for neighboring Baltimore County, Maryland, for the past 17 years. Baltimore County boasts the only Next Generation 911-ready PSAP in Maryland for the past six years. But what is even more interesting about the Baltimore County 911 infrastructure is that it leverages multiple, redundant systems to handle even the worst case scenario.
Resilience in Baltimore County’s 911 Center
Ray Windisch, Office of Information Technology (OIT) Director for the PSAP at Baltimore County, has a saying that he always recites to new vendors of the county:
“Baltimore County 911 never wants to be in the newspaper.”
Now, what he really means is that Baltimore County 911 ensures that their center is resilient enough so that there is never an outage that results in their 911 service being unavailable to the public. That bold statement dictates to the staff, the county executives, their vendors and partners their unwavering commitment to public safety.
To accomplish this level of resiliency, Baltimore County utilizes VoIP telecommunications technology currently used by modern contact centers. The software-based solution at Baltimore County is implemented in a distributed, geo-diverse, redundant configuration that supports their primary and backup PSAP centers. Each center is able to operate independently in case of a disaster at the other site, or together as a single, virtual center in the normal operational mode where staff can be at either site and can handle emergency and non emergency calls seamlessly.
The switching infrastructure operates in an “active-active” mode where both centers have live, operational telephone switches that can support both sites simultaneously and can support mobile and remote sites as necessary.
But this highly redundant and fault-tolerant solution is only one part of Baltimore County’s 911 system. Baltimore County also maintains a traditional digital telephone switching system as a backup in unlikely case that their redundant VoIP system fails. This legacy system, which originally served as the 911 system prior to their Next Generation 911 upgrade, is still fully functional and able to support Baltimore County 911 in case of a catastrophic failure of the VoIP system.
And, finally, as the last level of redundancy, Baltimore County also has the ability to flip a switch to route all 911 calls to analog phones in case all of their primary and backup systems fail. This last type of 911 call delivery is available to all PSAPs by the telephone network carrier in the case the PSAP equipment fails.
What sets Baltimore County 911 apart from other PSAPs is their commitment to periodically test all of their backup environments to ensure their staff is familiar with the transition process and to validate that the backup systems are still functional.
What Montgomery County (and Others) Can Learn from Baltimore County
The resiliency of the Baltimore County solution illustrates how with today’s contact center technology, redundant and fault tolerant 911 systems can be easily implemented. The key to implementing such an infrastructure is to hire a system integrator with experience in implementing such systems to design the 911 solution.
Had Montgomery County’s 911 center utilized these “best practices” when implementing their 911 solution, it is highly likely that such an outage that occurred to Montgomery County could have been avoided. Next Generation 911 technology, when implemented using techniques used by the best contact centers, allows for the fault tolerance required by 911 today.