Google Cloud MAJOR
Firestore degraded performance in europe-west3
September 27, 2022 · 08:02 AM UTC – 09:49 AM UTC · Duration: 1h 47min
Affected Services
Cloud Firestore
Timeline
07:44 PM
MINI INCIDENT REPORT
We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support.
(All Times US/Pacific)
Incident Start: 27 September 2022 00:02
Incident End: 27 September 2022 01:49
Duration: 1 hour, 47 minutes
Affected Services and Features:
Cloud Firestore
Regions/Zones: europe-west3
Description:
Cloud Firestore native and Firestore in Datastore mode customers experienced elevated error rates and latency for a duration of 1 hour, 47 minutes. From preliminary analysis, the issue was triggered by degraded query performance from a project causing an isolation issue and led to performance degradation in the europe-west3 region. The problematic project was isolated, mitigating the issue. Additional review is underway to prevent potential impact in the future.
Customer Impact:
Approximately 66% of projects in europe-west3 were affected.
Datastore native projects were not affected.
10:06 AM
The issue with Cloud Firestore has been resolved for all affected users as of Tuesday, 2022-09-27 02:00 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
09:33 AM
Summary: Firestore degraded performance in europe-west3
Description: We are experiencing an issue with Cloud Firestore beginning at Tuesday, 2022-09-27 00:02 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-09-27 02:15 US/Pacific with current details.
Diagnosis: Firestore native and Firestore in Datastore mode customers may experience high error rates and high latency
Workaround: None at this time.
09:13 AM
Summary: Firestore degraded performance in europe-west3
Description: We are experiencing an issue with Cloud Firestore beginning at Tuesday, 2022-09-27 00:02 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-09-27 02:00 US/Pacific with current details.
Diagnosis: Firestore native customers may experience high error rates and high latency for Firestore requests.
Workaround: None at this time.