Google Cloud MINOR
VMs using Local SSD are experiencing intermittent terminations.
October 27, 2022 · 06:32 AM UTC – 07:47 PM UTC · Duration: 157h 15min
Affected Services
Google Compute Engine
Timeline
07:47 PM
The issue with Google Compute Engine has been resolved for all affected users as of Wednesday, 2022-11-02 08:50 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
11:34 PM
Summary: VMs using Local SSD are experiencing intermittent terminations.
Description: We have completed all short term mitigation work to prevent impact to customers. VMs with Local SSD may still see a >0.008% chance of being terminated.
We have identified a fix for the remaining failures, and are working on a plan to roll out the fix, with our next update by Friday, 2022-11-04 16:00 US/Pacific.
Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine.
Workaround: Customers can reduce the possibility of being impacted by setting automaticRestart to false when scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false.
For VMs that are terminated, restarting the VM will restore the VM back to a working state.
11:44 PM
Summary: VMs using Local SSD are experiencing intermittent terminations.
Description: Mitigation work is currently underway by our engineering team.
We do not have an ETA for mitigation at this point.
We will provide more information by Friday, 2022-10-28 16:30 US/Pacific.
Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine.
Workaround: Set automaticRestart to false for scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false.
06:12 PM
Summary: VMs using Local SSD are experiencing intermittent terminations.
Description: Our engineering team continues to investigate the issue.
The impact of this issue was experienced beginning at Thursday, 2022-09-22 03:14 US/Pacific.
We will provide an update by Thursday, 2022-10-27 16:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine.
Workaround: Set automaticRestart to false for scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false.
09:20 AM
Summary: VMs using Local SSD are experiencing intermittent terminations.
Description: We are experiencing an intermittent issue with Google Compute Engine beginning at Friday, 2022-09-30 15:48 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Thursday, 2022-10-27 16:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine.
Workaround: Set automaticRestart to false for scheduling the VM under the host maintenance policy of a VM. Please see https://cloud.google.com/compute/docs/instances/setting-vm-host-options for details of the effects of setting automaticRestart to false.
06:40 AM
Summary: VMs using Local SSD are experiencing intermittent terminations.
Description: We are experiencing an intermittent issue with Google Compute Engine beginning at Friday, 2022-09-30 15:48 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Thursday, 2022-10-27 16:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine.
Workaround: Set the automaticRestart=false for scheduling the VM under host maintenance policy of a VM.
06:32 AM
Summary: VMs using Local SSD are experiencing intermittent terminations.
Description: We are experiencing an intermittent issue with Google Compute Engine beginning at Friday, 2022-09-30 15:48 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Wednesday, 2022-10-26 23:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers using Local SSD on VMs could see the VM terminated unexpectedly, which shows in the console as an Instance Termination by Compute Engine.
Workaround: Set the automaticRestart=false for scheduling the VM under host maintenance policy of a VM.