SDC CHAS - 10/28/2023

SDC CHAS - RCA

Outage - 0950 MST

Recovery - 1045 MST

Full Recovery - 1047 MST

The SDC CHAS location experienced a cascade error from within the Datacenter.    Feed Switch A Leg A failed on a power circuit at which time the 2nd power supply also failed causing a full path outage to the adjacent leg in the cloud.  The secondary switch then also failed.  This led to a partial collapse in the SDC CHAS location.  Upon further investigation the Switch collapse was traced to inefficient wiring in the original deployment leading to all power being delivered as A+A and not A+B.  This was restored by wiring A+B and moving the secondary back into a power position to perform power on restores.  The cloud self-recovered all services as expected. 

We will continue work within the DC to bring the primary to the same configuration and ensure via a physical audit that all legs, hypers, storage, switches and routers are accounted for in an A+B position for power.  We also verified fiber routes and connectivity and this is present and correct.  

No further follow on will be updated.

SDC Cloud Support