EMS Paging and Dispatch Servers Restored

The Zetron paging system was repaired by the company last week and returned to us. With Blumerich’s technical assistance it was brought back online late Friday afternoon. During the Zetron outage we set up the fire paging system to be able to tone out the EMS bases so they could still get notified for emergency runs. With that setup, we were only able to tone the bases and no backup personnel. Backup personnel are called out when all EMS rigs are on calls or otherwise unavailable. These personnel come into the base nearest the call for help and get an ambulance to respond.

With the system back the way it was before the lightning strike, we can tone the bases and backup personnel utilizing the alpha-numeric pagers and the Minitor V station pagers. EMS supervisors were unaffected during this time because they carry pagers operating on a cellular system from American Messaging which runs through the Internet. We have left the fire paging setup for EMS in place in the event that the Zetron goes out again. This way there will not be any setup time getting their station tones to go.

Additionally from the lightning strike, two dispatch servers’ Windows operating systems became corrupted. They didn’t crash until late Friday afternoon, the day after the storm. This can happen anytime there is a significant fluctuation in voltage to the server or any computer. Regardless of the tower and all connected equipment being properly grounded and on a UPS system (uninterruptible power supply) the lightning was more powerful than what the grounding could handle and obviously it ruined some things.

The first server to crash was the DSS Equature voice recording server. This machine records all telephone and radio traffic to and from the 9-1-1 center. Several attempts were made to restore the operating system using various repair software. They were unsuccessful and DSS was called in with a spare server. They changed out the servers and reconfigured the spare for our system and got recording back online. Unfortunately there were several hours where nothing was recorded.

Halfway through working on the DSS server the primary CAD (computer aided dispatch) server crashed. Thanks be to God for allowing us to successfully complete a prior project which gave us a backup CAD server with an instant failover software called Neverfail. When the primary server crashed, the backup server instantly assumed duty to become the primary CAD server. The dispatchers only noticed a small hiccup where they only had to restart the CAD application and continue working. Had we not had this Neverfail project complete, the dispatchers would be taking and dispatching calls by pen and paper. This method of dispatching is extremely inefficient and time consuming for this type of business. It wasn’t until today that the primary server was fully brought back online. The backup server has been running operations for the last two weeks. We don’t want to imagine our work being done on pen and paper for this long.

The lighting messed with the CAD server so much that its main board had to be replaced by an IBM technician. During the Neverfail project we signed a hardware and software maintenance agreement with IBM for $1,800 per year. It has already paid dividends as the board that was replaced had a cost of $3,000 and that doesn’t include the time and labor that the IBM tech had invested in the repair of the server. Once that was completed the RAID arrays (redundant array of independent disks) were reconfigured and the Windows operating system reloaded. After that, our CAD vendor VisionAIR had to load their software, manipulate the data from the backup server to the primary server and load and configure the Neverfail software again.