Tweet
My condolences about your outage. Based on the post on Hydrogenaudio, yet another case of backhoe fade, the plague of fiber optic cable systems. It looks like Netcetera and their provider have lost most of their "9"s.
Proper design calls for physical separation of redundant cables, divergent routing. Unfortunately, in all too many cases, that doesn't happen somewhere along the route, just the place where "dig we must". Quite often right in front of the facility, somehow both (or more) of the cables have to get into the facility, and that brings them close together. Steel conduit instead of the common PVC duct and/or encasing it in concrete helps; the backhoe operator is more likely to realize there is something in the way before ripping it out. But sometimes it is an unrecognized inadvertent intersection in the divergent routing. Back some years ago Verizon lost a whole string of it's central offices here, when the two redundant fibers for the control interconnection both ended directly underneath where they decided to plant a new pole. Oops, oh s!@*.
The broadcast network I worked for many years had, in the 1980s, recently switched from analog AT&T distribution of its radio network audio to digital satellite uplink. It all died one day less than a year later when, you guessed it, someone dug up both supposedly divergent fibers that carried the signal from their network operations center to the uplinker, right in front of their front door. After repair, they rerouted one of the cables out the rear of the building and down a different street.
At least this wasn't the most common cause of outages these days, a software/firmware update gone wrong. Yes, we had a few of those at the broadcast network, although (unlike various social media apps) I don't recall any of those actually knocking us off the air, at least before I retired. We were knocked off by more mundane causes, like failing UPS units, a contractor's cleaning crew accidentally hitting a switch shutting down the satellite uplink, snow in the uplink dish because the dish heaters had never been connected to a source of electricity(!!) and uncounted similar accidents. S!@* happens, it could be worse. (Although I hope your provider looks more closely at the routing of their supposedly divergent fiber routing).
Proper design calls for physical separation of redundant cables, divergent routing. Unfortunately, in all too many cases, that doesn't happen somewhere along the route, just the place where "dig we must". Quite often right in front of the facility, somehow both (or more) of the cables have to get into the facility, and that brings them close together. Steel conduit instead of the common PVC duct and/or encasing it in concrete helps; the backhoe operator is more likely to realize there is something in the way before ripping it out. But sometimes it is an unrecognized inadvertent intersection in the divergent routing. Back some years ago Verizon lost a whole string of it's central offices here, when the two redundant fibers for the control interconnection both ended directly underneath where they decided to plant a new pole. Oops, oh s!@*.
The broadcast network I worked for many years had, in the 1980s, recently switched from analog AT&T distribution of its radio network audio to digital satellite uplink. It all died one day less than a year later when, you guessed it, someone dug up both supposedly divergent fibers that carried the signal from their network operations center to the uplinker, right in front of their front door. After repair, they rerouted one of the cables out the rear of the building and down a different street.
At least this wasn't the most common cause of outages these days, a software/firmware update gone wrong. Yes, we had a few of those at the broadcast network, although (unlike various social media apps) I don't recall any of those actually knocking us off the air, at least before I retired. We were knocked off by more mundane causes, like failing UPS units, a contractor's cleaning crew accidentally hitting a switch shutting down the satellite uplink, snow in the uplink dish because the dish heaters had never been connected to a source of electricity(!!) and uncounted similar accidents. S!@* happens, it could be worse. (Although I hope your provider looks more closely at the routing of their supposedly divergent fiber routing).
Comment