Google Cloud London zone failed for six hours after power cut

Google Cloud said an electrical power interruption at the datacenter hosting availability zone europe-west2-b caused service disruptions, elevated error rates, API timeouts and processing latency across the europe-west2 region for 6 hours and 10 minutes on 20 October. The company's preliminary incident report gives the impact window as 07:00 to 13:10 BST (06:00 to 12:10 UTC) ^[1]. IT Channel Oxygen reported that Google said one of its three London availability zones suffered an incident that lasted from 07:08 to 15:00 UK time, describing it as a near eight-hour outage ^[3].

The outage disrupted banking apps, closed cafe tills and halted some patient appointment bookings, according to IT Channel Oxygen ^[3]. Firstpost reported that Revolut, HSBC and Lloyds were among the financial institutions affected ^[2].

Google's report says a power grid disruption occurred while backup generators failed to start because primary and backup programmable logic controllers did not initiate automatic start and transfer operations. The facility load ran on UPS battery banks until they were depleted, taking physical host servers, network switching and storage arrays in europe-west2-b offline ^[1].

Regional traffic was routed away from the offline zone, but failover traffic saturated compute slots in adjacent zones and control-plane API calls timed out waiting on the offline zone. Google said this caused intermittent HTTP 500 and 503 errors and admission queues across the region. Workloads kept only in europe-west2-b lost connectivity. Multi-zonal and regional services were degraded, including BigQuery, Looker, Compute Engine, Kubernetes Engine, Regional Persistent Disk and Cloud SQL ^[1].

Most Google Cloud Platform services recovered by 04:29 US/Pacific on 20 October (12:29 BST), 5 hours and 29 minutes after impact began. New VM creation resumed at 04:46 US/Pacific, and Google reported that healthy Compute Engine virtual machines exceeded 99 percent by 13:10 BST. Google posted closure at 07:03 US/Pacific (15:03 BST) ^[1].

The European Banking Authority contacted Google Cloud to obtain information and assess impact on EU financial institutions. The UK Financial Conduct Authority also engaged with Google, and the Treasury Committee asked the Bank of England and the FCA for details on the regulatory response, the scale of the disruption and any financial harm ^[2].

Revolut said the disruption affected logging in, card payments and cash withdrawals, and that customer funds and data remained safe. The bank is examining the event to reduce effects from future failures at external suppliers. HSBC and Lloyds did not comment ^[2].

The incident arrived as EU and UK regulators have tightened oversight of critical external technology suppliers. Google Cloud falls under both the EU regime covering 19 designated critical third-party providers and the UK regime covering four cloud providers. The UK framework requires designated suppliers to report serious outages to the FCA and Bank of England ^[2].

Tim Pfaelzer, senior vice president EMEA at Veeam, said the outage showed how much UK daily activity depends on a few cloud platforms, and argued businesses should define a 'Minimum Viable Business' that identifies the core processes, systems and data required during an outage ^[3]. Justin Sharrocks, general manager EMEA at TrustedTech, said resilience planning should follow business impact and that plans need to be tested before an incident ^[3].