Google Cloud MAJOR

Firestore degraded performance in europe-west3

September 27, 2022 · 08:02 AM UTC – 09:49 AM UTC · Duration: 1h 47min

Affected Services

Cloud Firestore

Timeline

07:44 PM
MINI INCIDENT REPORT We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support. (All Times US/Pacific) Incident Start: 27 September 2022 00:02 Incident End: 27 September 2022 01:49 Duration: 1 hour, 47 minutes Affected Services and Features: Cloud Firestore Regions/Zones: europe-west3 Description: Cloud Firestore native and Firestore in Datastore mode customers experienced elevated error rates and latency for a duration of 1 hour, 47 minutes. From preliminary analysis, the issue was triggered by degraded query performance from a project causing an isolation issue and led to performance degradation in the europe-west3 region. The problematic project was isolated, mitigating the issue. Additional review is underway to prevent potential impact in the future. Customer Impact: Approximately 66% of projects in europe-west3 were affected. Datastore native projects were not affected.
10:06 AM
The issue with Cloud Firestore has been resolved for all affected users as of Tuesday, 2022-09-27 02:00 US/Pacific. We thank you for your patience while we worked on resolving the issue.
09:33 AM
Summary: Firestore degraded performance in europe-west3 Description: We are experiencing an issue with Cloud Firestore beginning at Tuesday, 2022-09-27 00:02 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2022-09-27 02:15 US/Pacific with current details. Diagnosis: Firestore native and Firestore in Datastore mode customers may experience high error rates and high latency Workaround: None at this time.
09:13 AM
Summary: Firestore degraded performance in europe-west3 Description: We are experiencing an issue with Cloud Firestore beginning at Tuesday, 2022-09-27 00:02 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Tuesday, 2022-09-27 02:00 US/Pacific with current details. Diagnosis: Firestore native customers may experience high error rates and high latency for Firestore requests. Workaround: None at this time.