Fibre Break Affecting Bolton Stone Cross, Bolton

Event Started
2022-08-25 13:33
Report Published
2022-08-25 13:59
Last Updated
2022-08-26 20:16
Event Finished
Ongoing

Damage to a backbone fibre serving a large number of service provider and business networks is affecting connectivity in Bolton. Repair teams are working through the night to fix the broken cables.

Timeline (most recent first)
  • 2022-08-26
    16:57:00

    We can confirm all splicing is complete and no further work will be performed on the cables.

    Engineers have completed a full and permanent repair to all necessary cables and protection. All measures are taken to ensure our fibres are protected - as with all permanent repairs.

    The 3rd party contractors are aware of the impact they have caused and to proceed with additional caution. Further investigation will continue in a post incident review.

  • 2022-08-26
    05:25:00

    On the first 288f cable, there are 5 tubes left to splice at both ends.

    On the second 288f the A-End is now completely spliced and there are 14 tubes left on the B-End.

    Once both 288f cables have been spliced, the ERS Splicers will then begin preparation work on the 1x 144f and 1x 12f cables.

    Plan:

    • ERS Splicers to complete splicing for the 2x 288f cables.

    • Once 288f cables have both been spliced, ERS Splicers to begin prepping and splicing of 1x 144f and 1x 12f cables.

    Team currently on-site:

    • 2 Network Engineers
    • 4 ERS Splicers
    • Traffic Management
    • Civils Crew
    • Cable Gang Further updates to follow.
  • 2022-08-26
    04:00:00

    As of 03:05 UTC, confirmation that on the first 288f cable, there are 8 tubes (of 12 fibres) left to splice at both ends.

    On the second 288f there are 16 tubes (of 12 fibres) left on the A-End and 20 tubes left on the B-End.

    Plan:

    • ERS Splicers to complete splicing for the 2x 288f cables.

    • Once 288f cables have been spliced, ERS Splicer to begin prepping and splicing of 1x 144f and 1x 12f cables.

    Team currently on-site:

    • 2 Network Engineers
    • 4 ERS Splicers
    • Traffic Management
    • Civils Crew
    • Cable Gang
  • 2022-08-26
    02:28:00

    Service for Stone Cross is back online!

    Meanwhile, splicing teams are continuing to work to repair the rest of the fibres in the broken cables.

  • 2022-08-26
    01:41:00

    Services should steadily be seen to restore on the first of the 288-fibre cables.

    2 Splicers have finished preparation work at both ends of the first 288f cable and have commenced splicing. The 3rd Splicer has finished preparation work and has commenced splicing at the A-End of the 2nd 288f cable. The 4th Splicer is still prepping the B-End of the 2nd 288f cable. As both 288f cables hold the vast majority of impacted Business customers, these cables will be prioritised, before moving onto the 144f and 12f cables.

  • 2022-08-25
    21:21:00

    Work is ongoing, but as yet we have no estimated time of resolution.

    Network Engineers have confirmed that 2 further ERS Splicers are now on site to assist with resolving this incident. Confirmation was also received that the Civils Crew have also arrived on site and have commenced work to introduce the bottle joints, within the existing joints, either side of the break. An ETA for the cable gang along with replacement 2x 288f, 1x 144f and 1x 12f cables is being sought and escalations are being made. As previously mentioned, the current fix plan is for 8 new joints to be introduced in existing joints either side of the break. Once introduced and on site, the Cable Gang will proceed with pulling in the replacement cables, before prepping and splicing can commence. We have been advised that due to the extent of the damage, that this will not be a quick fix and restoration will more than likely run over in to tomorrow.

  • 2022-08-25
    19:27:00

    Over 700 fibres have been damaged by civil engineering contractors, and need to be replaced and re-jointed. We have been advised that "this will not be a quick fix and restoration" and it is likely to run into tomorrow. We do not have an estimated time of resolution at this stage.

    Confirmation has been received that 2x 288f, 1x 144f and 1x 12f cables have been completely damaged in the joint on Barton Road, Manchester. The damage has been caused by 3rd party civils who have been working in the area and joint today. The fix plan is for 8 joints to be introduced to allow for the replacement cables to be introduced before being prepped and spliced. Confirmation has been received that Traffic Management are now on site and will divert road traffic in order for the Cable Gang and Civils Crew to safely work on site. 2 ERS Splicers are on site and 2 further ERS Splicers have been requested to attend site to assist.

  • 2022-08-25
    18:49:00

    The latest update we've received translates as "someone was digging up Manchester Road, and they cut a backbone fibre".

    Another Network Engineer is currently relocating to site to assist with the outage. 3rd party civils work has also been identified as potentially ongoing in the same area. As an OTDR event has been detected, a confirmed fibre break has been declared and once the engineer attended the site, located the fibre break and could see visible damage to VMO2 owned fibre cables. Network Engineers are currently reviewing the damaged cables within a trench dug by 3rd party civils. Further ERS request has been created to request 2x ERS Splicers, Cable Gang and Civils with Traffic Management to attend Manchester Road.

  • 2022-08-25
    17:20:00

    The ERS, Civils and two splicers are currently on route and we are waiting for an ETA.

  • 2022-08-25
    16:13:00

    This looks to be a complex fault affecting a large number of fibres.

    The network engineer has tested the fibres and identified a possible fibre break. They are currently in the process of engaging ERS splicers, cable gang and civils teams.

  • 2022-08-25
    15:43:00

    Following initial investigations an engineer is now on site investigating the fault at a Manchester hub site, currently testing fibres.

    We will provide a further update as soon as we have further information.

  • 2022-08-25
    14:23:00

    Our fibre engineers have confirmed a fault, and a priority 1 incident has been raised.

  • 2022-08-25
    14:02:00

    Our fibre engineers have confirmed there are a large number of incidents that occurred around the same time, confirming our suspicions of a fibre break. We are waiting for their next update in a few minutes' time.