Google Cloud MAJOR
Google engineers are investigating a global issue with Cloud Memorystore.
December 20, 2022 · 09:42 AM UTC – 07:15 PM UTC · Duration: 9h 33min
Affected Services
Cloud MemorystoreMemorystore for Redis
Timeline
12:02 AM
Mini Incident Report
We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support .
(All Times US/Pacific)
Incident Start: 15 Dec 2022 18:30
Incident End: 20 Dec 2022 09:54
Duration: 4 days, 15 hours, 24 minutes
Affected Services and Features:
Cloud Memory Store: GUI Instance Provisioning
Regions/Zones: Global
Description:
Cloud Memory Store users experienced sporadic levels of inability to create new instances from 15 Dec 2022 at 18:30 PT until 19 Dec 2022 16:00, when the failure rate reached 100%. The root cause for this issue was an errant configuration rollout.
Customer Impact:
Affected customers were unable to create new instances, as the region field in the user interface (UI) could not be populated due to a metadata issue.
Affected users were able to work around the issue by using the “gcloud redis instance create” command to provision an instance and the gcloud redis regions list.
eg “gcloud --project=<project_name> redis instances create <instance_name> --size=1 --tier=<basic/standard> --region=
07:15 PM
The issue with Cloud Memorystore has been resolved for all affected users as of Tuesday, 2022-12-20 11:14 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
05:22 PM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: Mitigation work is currently underway by our engineering team.
The mitigation is expected to complete by Tuesday, 2022-12-20 11:30 US/Pacific.
We will provide more information by Tuesday, 2022-12-20 11:30 US/Pacific.
Diagnosis:Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: Customer can use gcloud redis instance create command to provision an instance and the gcloud redis regions list. eg gcloud --project=<project_name> redis instances create <instance_name> --size=1 --tier=<basic/standard> --region=
03:41 PM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: Mitigation work is currently underway by our engineering team.
The mitigation is expected to complete by Tuesday, 2022-12-20 08:45 US/Pacific.
We will provide more information by Tuesday, 2022-12-20 09:30 US/Pacific.
Diagnosis: Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: Customer can use gcloud redis instance create command to provision an instance and the gcloud redis regions list. eg gcloud --project=<project_name> redis instances create <instance_name> --size=1 --tier=<basic/standard> --region=
11:29 AM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: Mitigation work is currently underway by our engineering team.
The mitigation is expected to complete by Tuesday, 2022-12-20 08:45 US/Pacific.
We will provide more information by Tuesday, 2022-12-20 09:30 US/Pacific.
Diagnosis:Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: Customer can use gcloud redis instance create command to provision an instance and the gcloud redis regions list.
11:28 AM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: Mitigation work is currently underway by our engineering team.
The mitigation is expected to complete by Tuesday, 2022-12-20 04:45 US/Pacific.
We will provide more information by Tuesday, 2022-12-20 09:00 US/Pacific.
Diagnosis:Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: Customer can use gcloud redis instance create command to provision an instance and the gcloud redis regions list.
10:52 AM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: We are experiencing an issue with Cloud Memorystore beginning at Tuesday, 2022-12-20 00:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-20 04:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis:Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: Customer can use gcloud redis instance create command to provision an instance and the gcloud redis regions list.
10:36 AM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: We are experiencing an issue with Cloud Memorystore beginning at Tuesday, 2022-12-20 00:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-20 04:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis:Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: Customer can use gcloud redis instance create command to provision an instance and the gcloud redis regions list.
09:55 AM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: We are experiencing an issue with Cloud Memorystore beginning at Tuesday, 2022-12-20 00:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-20 03:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: None at this time.
09:54 AM
Summary: Google engineers are investigating a global issue with Cloud Memorystore.
Description: We are experiencing an issue with Cloud Memorystore beginning at Tuesday, 2022-12-20 00:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-20 02:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers are unable to provision a new instance using Google Cloud Console, the region field will not populate.
Workaround: None at this time.