This past week we experienced another breakdown in the communications system. This time it is was on the voice side. At the Imlay City tower site voice channel bank number 5 had a power supply and power amplifier failure. Blumerich replaced both parts with new ones after having Motorola ship them with overnight delivery.
The good news out of this is that although there was a failure of equipment, no one noticed that there was a problem. This is because the voice system has plenty of capacity for traffic and although one of the voice banks went down, the system still had excess capacity to handle all of the radio traffic. The only time a failure like this would become noticeable would be if there was a huge influx of users in the area of the affected tower.
In the case of the Champion Bus fire a few months ago, had we lost a voice bank or two, there could have been a possibility of users getting a “bonk” when they try to transmit, indicating the system is too busy at that particular moment to handle their transmit request. The user would just have to keep trying every few seconds to get permission to make a transmission.
The power supply and power amplifier both are covered under our Motorola service contract.
So far in 2010, we are seeing more frequent breakdowns of various equipment at each of the sites than we have in the past. The age of the system is quickly catching up to it.
Due to heightened sensitivity of the issues surrounding the paging system I am hearing reports of strange things occurring on folks pagers. Things such as one department being toned and a pager from another department opening up but not hearing any voice, pagers opening up when no one is being toned etc.. Many of these occurrences are not new and have been happening to some degree over the years and I would like to say a few things about that.
First, the Minitor V is a very sensitive device and has a history of opening up on “close” tones. Many of the departments in Lapeer County have tone sets that are very close to one another and on occasion will open up when it thinks it hears its own tone. A few years ago Motorola issued a software update to help control sensitivity of the device. If you think you have a pager that is still too sensitive you can bring it in and I can further adjust the sensitivity of it.
There is also the likelihood that there could be some other transmitting device in a frequency band near our assigned frequency that could be transmitting at high power and causing bleed over into our band and causing a pager to open up. As an example, I’ve had it happen before on my car stereo while listening to the radio that a person on CB with a linear amplifier (which is illegal to use) was pushing so much power on the citizens band that his transmission bled over the FM radio band and blasted me in my car right over the radio station.
Secondly, there are areas of the county that traditionally have poor coverage even though they are in relative close proximity of a tower. The likely cause for this is what is called “dead zones” in radio wave propagation combined with terrain features. This also accounts for why people will be well outside of the county and get a page and have it be crystal clear.
Thirdly, for some who keep their pager on an “open” status, on occasion you hear bursts of data. Those are a couple of different things. One kind is the alpha-numeric pages that Lapeer County EMS uses and formerly some MFR groups. Also the system itself sends out data bursts periodically to “poll” the sites. The system polls the sites to make sure they are either still online, offline or to find out if there is something going wrong that it needs to send out an alarm about.
There are just so many variables to look at in wireless communications from terrain features, vegetation, local weather conditions, space weather (i.e. solar flares, sun spots), buildings, orientation of the pager, where it is worn on the body, body type, direction the pager (or radio) is facing relative to the source signal, heavy machinery, high tension power lines, electronics and so on. All of these things and more play a role in how radio communications work, or don’t work for that matter. I think that many times, these weird things that happen are not always due to a problem with the system, but can be attributed to external factors interfering with our system.
Also important is the age of our system as we are all very aware of. Blumerich and I keep a very close eye on the system and we check it and test it weekly to make sure it is operating within specification. Even though when it is operating normally, there can still be issues with receiving pages that just cannot be explained other than looking at some of the other variables I mentioned already.
Regardless of all that, the looming age of the system is a big concern and every time we experience a problem with it, it further highlights the need to have it replaced with a new system and more towers installed to increase coverage in the deficient areas. Please continue to keep your ears tuned to any potential problems and let us know about them.
This week Blumerich Communications (our radio shop) installed a power supply for what we call the “page bridge” which is the unit that failed in March that brought the entire system down. When that happened we installed the only spare power supply we had to bring the system back up and that left us without a spare backup.
Blumerich found another one on eBay and purchased it as is. They tested it and it seemed to work. Then they brought it out this week and installed it to make sure it worked on our system and it did. It ran a little hot (high voltage) but within specification. They took it back out and tuned it down and then put the original power supply back in.
So right now we have a spare power supply again for the page bridge. As we are well aware, it is a used part of equal age of the system and has no warranty. Therefore when and if it is needed, there is no guarantee that it will work for long. But at least we have a backup unit again.
Also we ordered two new power supplies from Motorola for the nucleolus that failed in Lapeer last weekend and we’ll have them in hand on Tuesday.
On Saturday, June 19, the paging system for fire departments and Lapeer County EMS partially went down in the area of the Lapeer tower. The Imlay City and North Branch tower areas were unaffected. It was a combination problem of a faulty power supply and a circuit board called a nucleolus, which is the “brain” of the paging system. Each site has its own nucleolus, which allows them to continue to operate though one site is down. Fortunately we had several spare power supplies and one spare nucleolus.
We felt due to the concentration of population surrounding the Lapeer tower as opposed to the North Branch tower that we were going to pull the nucleolus from North Branch and put it in service in Lapeer to get paging back online in the area. Blumerich Communications (our radio shop) went to North Branch and pulled the parts and brought them to Lapeer. The boards proved to be a bit different from each site and when inserted wouldn’t work right. By all indications, it was only the nucleolus that was bad as both power supplies were working and showing faults, and the two cards (the nucleolus and another card) were not coming up. Working with several cards and different technical methods took quite some time. We found that the first spare power supply we pulled out of storage turned out to be a bad unit and didn’t work. We were able to insert the original nucleolus into the Lapeer site with a second spare power supply and that brought it back online.
During this time, it was discovered that the nucleolus from the North Branch site was going bad as it would only power up and then go into a reset loop. By this time it was 2am. Earlier in the evening dispatch made phone calls to all the departments advising them of the situation. Sunday morning we returned after getting some rest and took the spare nucleolus back to North Branch. The spare nucleolus needed programming for our system and Blumerich had copied the proper settings from the working board in Lapeer before going to North Branch. Once the spare was programmed and inserted into the system we did a test page with Burlington Fire and people in Tuscola County and Marlette as well as in Lapeer received the page. After the successful page with Burlington it was confirmed that all sites were back online. We directed an ‘all call’ page for the entire county around 1200 on Sunday to alert the fire service that the system was back online at 100% for all three sites.
Additionally, all the Lapeer County EMS bases were on radio standby initially and then only the North Branch EMS base was required to be on radio standby, once the problem was diagnosed. Now all the bases are back on routine paging for calls.
The nucleolus board is an obsolete part and went off the Motorola contract several years ago. Had we not had a spare on hand, it is unlikely that we would be able to get another one or get one repaired. We are going to try and find another one, as well as try and repair the bad one. The nucleolus is a critical part of the paging system.
Coincidentally, earlier in the day Saturday we had a high temperature alarm in the Imlay City tower site. The A/C had gone out and the threshold of heat inside the radio room was exceeded. We contacted the County Buildings and Grounds department and they went out and fixed the problem. Though it was a high temperature, the systems were unaffected. This problem was completely separate and unrelated to the paging problem.
Last week there were rumors that the 9-1-1 lines were down. That is a false rumor. What really happened was that a fiber optic line owned by Verizon was accidentally cut by a construction crew. That knocked out telephone service in the south end of the county to the businesses and residents there. It only affected landline users in that they couldn’t call out or receive calls, including 9-1-1.
The local media was contacted to let people know that their phones may not be working. This outage did not have to do with the actual 9-1-1 lines. People in the south end of the county with cell phones were still able to contact us.
Lapeer County Central Dispatch has not been slack concerning the need for a replacement system and grant processes. The process of seeking a replacement system and it’s requirements began in August of 2007 when Motorola notified us that the microwave and paging systems would no longer be supported under contract effective January 1, 2008 and repairs would be based upon parts availability. The site controllers which are the brains of the radio system would no longer be supported after July 1, 2009. The Radio Network Controller which runs the in-car police computers would not be supported after January 1, 2010. The original portable radios would not be supported after June 2010 and the original mobile radios would not be supported after September 2010.
Immediately upon notification from Motorola that the systems were starting to be obsolete LCCD began the process of hiring an independent consultant, free from any ties to any radio system company, to come in and evaluate our systems and give us a recommendation on what type of system to go to for replacement. That process started in August 2007, the same month Motorola gave us their notification.
Kimball & Associates was hired to be the consultant and came out several times to see the equipment, our operations, our needs and to meet with the members of the police, fire and EMS agencies which utilize our system. Kimball also met with the Authority and Technical Advisory Committees several times during the evaluation process. Their findings and recommendations were presented to the LCCD Authority & Technical Advisory Committees in March 2008 and then again to the County Board of Commissioners at a summer meeting in Goodland Township on July 17, 2008.
Kimball, based on their findings and our needs, recommended that LCCD seek to replace the aging system with the Michigan Public Safety Communications System (MPSCS). This was the most economically viable choice that would fulfill the needs of public safety in Lapeer County. The other choices presented inadequate coverage and safety concerns. The LCCD Technical Advisory Committee reviewed and explored the options presented by Kimball and in December 2008 came to a consensus on moving to the MPSCS as Kimball recommended.
In January 2009 a sub-committee was formed to explore funding options for the recommended replacement system. This committee included members of the LCCD Technical and Authority Board as well as officials from the County of Lapeer. Many ideas were explored and a county-wide millage was deemed the most economically feasible and fair way to fund a replacement system. This sub-committee was given a ballpark figure from Motorola on what a replacement system would cost. In June 2009 a budgetary quote was requested from Motorola and it was received in July 2009. By August 2009, LCCD brought this budgetary figure along with a business plan, coverage studies and other facts and figures to the County Board of Commissioners requesting a special election for November 2009.
There are grant monies available to public safety agencies but not all are available to LCCD. For example there is the Fire Act Grant with is exclusively available to fire departments. The COPS grant is also only for law enforcement agencies and not open to LCCD or other 9-1-1 centers. There are areas of Homeland Security money available to LCCD.
We have applied for federal grant money (stimulus), but one of the stipulations was that projects had to be “shovel ready” and ours was not at that stage at the time. We have been in direct contact with Congresswoman Millerʼs office who is watching for the next batch of available grants that we can apply for.
Grant money is distributed from the federal government to the individual states. In Michigan, that money is then sent down to regions where committeeʼs are formed to decide how to spend the money. Lapeer County is one of 14 counties in our region. Grant money is not sent directly to the county to be used for anything we want. Unfortunately, grant money is not like our individual savings accounts. No one can apply for grant money and then put it aside and save it up until a purchase is ready to be made. The rules regarding how grant money is used are very specific and do not allow for saving it up among many other things. When applying for any grant, one must have a specific item(s) already picked out, the details of it sent in with the grant application and how it is to be used. There usually comes with any approved grant request a 1 to 3 year audit trail as well.
In addition to that, grants are competitive, meaning, that all 14 counties in our region must compete with each other for money out of the same pot. Oneʼs counties needs are weighed against that of another counties needs, and point systems are developed to grade those needs. Once everything is worked out, the grant requests are then sent back to the state for approval. This is just a quick summary of how it works, and also know that these processes take months to years to complete before any county or agency would actually come into possession of their grant request. The cycles are always years behind.
No one entity can apply for such a large sum of money as to fund the needed replacement of our systems in any grant. Typically, at the federal level that kind of thing is called an ʻearmarkʼ. Earmarkʼs are not permitted in the grant process. The hundredʼs of millions of dollars available from the Department of Homeland Security can be misleading. That money is spread out across the country to the states as mentioned already and then down to the regions. Within the states and regions, the money is then divided up into subcommittee’s with different needs and objectives. Not all of the money goes to communications. For example our region is about finished up with the 2007 DHS grant for communications and it only had approximately $3 million dollars to spend between 14 counties. that is the largest sum of money to come though the region. Surcharge money is used to cover our operational costs. Currently it brings in just enough money to keep the doors open.
The grant money that has been awarded to Lapeer County over the years has gone to purchase many new hybrid radios capable of both analog and digital systems, new in-car computers for law enforcement, and bi-directional antennaʼs to bolster the MPSCS signal in the basement of the Emergency Operations Center. It has also purchased the countyʼs mobile command vehicle and equipped it with the necessary items for all disciplines of public safety to use in times of major local emergencies. All of these things have already supplanted the cost of the proposed replacement project.
Through the Emergency Management Department law enforcement obtained 44 radios in 2004 and 53 radios in 2005. The Warren Grant of 2004 garnered 19 mobile radios, 63 spare batteries, 53 remote speaker microphones, 54 chargers and 3 multi-chargers. Grants were also obtained through a regional medical committee in 2006 to purchase radios for EMS agencies (including medical first responder teams). Also in 2004 through the Department of Homeland Security, Lapeer County Central Dispatch acquired a grant for a digital mapping system along with aerial photos of the whole county for a cost of $184,440.80.
Stimulus money for 2010 was applied for through Congresswoman Millers office in February 2010 for the full amount of the replacement system, though it is not expected to receive any or all of it.