Pre-Event Mitigation

If you determine that you have devices which may fail during the rollover event, there are certain steps you can take ahead of time to either reduce or eliminate the symptoms of the event. We recommend that everyone with definitely affected (Revision J) devices take one or more of these steps. For devices which may be affected, one should balance the impact of an unlikely (but possible) failure with the impact and hassle of taking these steps ahead of time.

Where appropriate setting exists, disable GLONASS

In the latest firmware in our PowerInjector and SiteMonitor Base 3 products, you can select which GNSS constellations you use. By default, these devices configure our latest SyncBoxes to use only GPS+Galileo which eliminates the underlying issue, since GLONASS is no longer enabled.

As mentioned, this functionality is only in recent firmware. Once you upgrade to a version which has this setting, simply check that the constellation is set to GPS+Galileo or GPS only. On the RackInjector you only need to set it once. On the Base 3, you'll need to set it on all of the attached SyncInjectors or PowerInjectors.

The setting itself is found in the gear menu on the GNSS Receiver Status section of the GNSS and 1PPS menu.

Set radios to operate for some period after loss of sync

As the most likely outcome of a affected GNSS receiver is failure of the 1PPS sync pulse, setting radios to continue to run without the sync pulse being present is a good way to buy yourself time after the rollover to be able to reboot affected radios.

On the 450 platform, this setting is found on the Configuration->General Tab. The Sync Input should be set to "AutoSync+FreeRun".

Enabling the setting to generate sync before a pulse is received could reduce downtime but is likely to introduce self-interference in the event a radio is rebooted. We do not have a strong recommendation as to what this setting should be set to for this event as we can make strong arguments for either setting.

The ePMP platform has a similar setting on the Configuration->Radio page. It is entitled "Synchronization Holdoff Time". We recommend setting this setting to the max which is 86400 or one day.

The above settings can be returned to your preferred settings early in 2021.

Ensure adequate monitoring is in place

If any GNSS receivers in your network are affected by the rollover event, then being able to locate them quickly to reboot them is important. As a result, we recommend that our customers verify that any monitoring system they may have in place is configured to monitor the status of the GNSS system. Usually this is as easy as monitoring the Synchronization status OID in the radios themselves.

Make sure you are able to remotely reboot any receivers which are likely to fail

On the recovery page, we provide instructions for common ways to power cycle a failed GNSS receiver. We recommend that our customers pay attention to each revision J receiver in their network and try to ensure that one of the methods is available to them to reboot the device in the event it needs to be rebooted.

Note that in 2020 some devices seemed to recover on their own, while others needed a reboot. Many customers who had hard to reach Revision J receivers found that by ignoring the problem for an extended period that the device simply recovered. Please note that we are not stating that the units will for sure recover without a power cycle or reset. Instead, we're stating that in some cases waiting to see if a GNSS receiver recovers on it's own might be a viable option where it isn't reasonable to reset the unit remotely.

Be aware of the exact time of the rollover and be prepared to recognize any failures which may occur

The rollover occurs at Midnight, Moscow Time. An easy way to determine what time of day this is for a local time zone is to query google with the query: "Midnight Moscow time to MST", where MST is the abbreviation for your local time zone. For our North American customers, this equates to 4PM eastern, 3PM central, 2PM mountain and 1PM pacific standard time. All on December 31st.

In 2020, customers started seeing failures almost immediately on affected devices. These failures seemed to consist of one or more of the following:

• Dates which were reported to be four years earlier
• Inability to maintain a GPS lock on satellites (i.e. an immediate drop in satellites available)
• Loss of GPS sync

Failed receivers either recovered on their own or needed a power cycle. After a power cycle, the devices started to work normally and have been generally trouble free since.