Power Issue at Equinix MA1 Backbone

Event Started
2022-05-24 09:01
Report Published
2022-05-24 09:25
Last Updated
2022-06-20 08:53
Event Finished
2022-05-24 09:11

We're aware of a power blip affecting multiple carriers in Equinix MA1. Our circuits have since restored, but we don't know what the cause has been.

Timeline (most recent first)
  • 2022-06-20
    08:51:00

    Equinix IBX Site Staff reports that battery Core 2 of the UPS has been brought back online after the successful commissioning of the Core 2 battery bank. The system is currently stable and will be monitored for further stability.

    Situation Status: Stable

    Equipment/Condition: Power Disturbance - Customer Loads on Utility Power

    Current System Redundancy: Normal

    Resources on Site/Pending: Vendor Arrived

    Generator Fuel Run Time (hrs.): N/A

    Current Colo Temperature Status: Temperature Stable, within SLA

    Current Colo Humidity RH Status: Humidity Stable within SLA

  • 2022-05-27
    11:12:00

    Equinix IBX staff reports that customer loads remain stable on utility power and UPS backed at N+1. Core 2 remains off line and we are working with our specialist vendors to accelerate our scheduled battery replacements that were due to take place on the 20th June 2022. Once the battery replacement works are complete and the issue has been resolved.

  • 2022-05-26
    17:42:00

    Equinix IBX Site Staff reports that they started battery bank commissioning. The Customer load will remain on the utility power supply and UPS System backed at N+1 throughout the remedial works.

  • 2022-05-24
    15:30:00

    Another update:

    Equinix IBX Site Staff reports that IBX Critical Facility Engineers and the vendor have successfully transferred all customer loads from the generator power back to the utility source, where they remain stable. The UPS system is at N+1 redundancy, however, Core 2 of the UPS remains isolated. IBX Critical Facility Engineers and the vendor are sourcing a replacement battery. The situation is under observation.

  • 2022-05-24
    14:03:00

    We have received the following update:

    Equinix IBX Site Staff reports that the battery measurements were performed on Core 1 , 3 and 4 and all were confirmed healthy. The faulty battery string on Core 2 remains isolated and under investigation. Customer loads remain stable on generator and will be transferred to UPS backed Utility Power at approximately 15:00 Site Local Time. No impact is expected to customer equipment during the power transfer to the utility

  • 2022-05-24
    09:11:00

    Power restored, and leased line connectivity passing through MA1 restored within 10 minutes.

    We are awaiting further information as to the cause and impact.