laitimes

Google Cloud one-click database deletion: affecting more than 500,000 users, crashing for a week, and $125 billion in jeopardy

author:DBAplus Community

Recently, Thomas Kurian, CEO of Google Cloud, and the head of UniSuper, an Australian non-profit superannuation fund, jointly issued a statement to apologize to UniSuper's 620,000 members for the incident of accidental deletion of UniSuper private cloud accounts due to the "misconfiguration" of Google cloud services.

Google Cloud one-click database deletion: affecting more than 500,000 users, crashing for a week, and $125 billion in jeopardy

As a result of the failure, more than 500,000 members of the UniSuper Fund were unable to access their superannuation accounts for an entire week from May 2. Although the service has been gradually restored since Thursday, the balance data of the investment account still needs to be updated to reflect the amount of the previous week.

UniSuper's CEO Peter Chun sent a letter to members on Wednesday evening to make it clear that the outage was not caused by a cyberattack and that there was no personal data leakage during the outage. He pointed to Google Cloud Services as the cause of the problem.

In a joint statement, Kurian and Chun highlighted the isolated nature of the incident and confirmed that Google Cloud had identified the cause of the outage and had taken steps to prevent a similar incident from happening again. They said it was a "one-of-a-kind incident" that had never been encountered before by Google Cloud's customers around the world.

Although UniSuper typically has copies of its data in both regions, cloud instances in both regions were affected due to the deletion of cloud subscription accounts. Eventually, UniSuper was able to resume service thanks to a backup from another vendor. These backups minimize data loss and improve UniSuper and Google Cloud's ability to complete recovery.

"The dedication and collaboration between UniSuper and Google Cloud has enabled us to recover our private cloud extensively, including hundreds of virtual machines, databases, and applications."

Google Cloud one-click database deletion: affecting more than 500,000 users, crashing for a week, and $125 billion in jeopardy

In the "Update on the Situation" announcement on the UniSuper website on May 10, "we have made tremendous progress in bringing all systems back online. Members can log in to their accounts online to check their account balances and view the latest investment performance of their accounts. ”

UniSuper manages about $125 billion in funding, and the outage has raised widespread concern and concern in the industry, as well as raising questions about the security and stability of global cloud services. Google Cloud's misstep as the world's leading cloud service provider had a significant impact on its reputation.

Google Cloud one-click database deletion: affecting more than 500,000 users, crashing for a week, and $125 billion in jeopardy

The incident is also a reminder to cloud service users around the world of the importance of data security and business continuity plans. With the popularization of cloud services, how to ensure the stability and security of services has become a common challenge for all cloud service providers and users.

In response to this incident, cloud service providers and users may be able to make improvements in these areas:

  • Strengthen backup mechanisms: Enterprises should establish a comprehensive backup strategy, including regular backups, offsite backups, and cloud backups. Backups should be tested regularly to ensure that they can be recovered quickly in the event of data loss. In addition, backups should be stored in a different physical location from the original data to avoid a single point of failure.
  • Strict access control: Access to cloud services must be strictly managed to avoid unauthorized access and operations. The principle of least privilege should be used to ensure that employees only have the access necessary to do their jobs. At the same time, for critical operations, such as data deletion, a two-factor authentication mechanism should be implemented.
  • Change management process: Any configuration changes that may impact data integrity and availability should follow a formal change management process. This includes upfront change impact assessments, change documentation, approval processes, and post-event change audits.
  • Real-time monitoring and rapid response: Implement a real-time monitoring system to be able to detect and respond immediately in the event of data loss or service disruption. The monitoring system should be able to cover all key cloud service components and be configured with automatic alerting mechanisms.
  • Service Level Agreements (SLAs) and Contract Terms: When signing a contract with a cloud service provider, you should specify the service level agreement (SLA), including terms such as service availability, data protection, recovery time, and compensation for breach of contract. At the same time, the contract should specify the boundaries of data ownership and liability.
  • Employee security training: Conduct regular security awareness and best practice training for employees, especially for those who have access to operate cloud services. The training should include data protection, identifying phishing attacks, security configuration, and emergency response.
  • Disaster recovery and business continuity plan: Develop a detailed disaster recovery plan and business continuity plan to ensure that in the event of a major failure, you can quickly switch to a backup system with minimal impact on your business.
  • Data classification and sensitivity management: Classify data stored in the cloud and take appropriate security measures based on its sensitivity and importance. For highly sensitive data, a higher level of encryption and access control should be employed.

Organize the dbaplus community

The DBAPLUS community welcomes contributions from technical personnel at [email protected]