Google Cloud MAJOR
Degraded serving for Google App Engine and Cloud Functions users in us-central1
May 24, 2022 · 03:01 PM UTC – 03:50 PM UTC · Duration: 49min
Affected Services
Google App Engine
Timeline
07:36 PM
We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case https://cloud.google.com/support or help article https://support.google.com/a/answer/1047213.
(All Times US/Pacific)
Incident Start: 24 May 2022 07:01
Incident End: 24 May 2022 07:50
Duration: 49 minutes
Affected Services and Features:
App Engine
Cloud Functions
Regions/Zones: us-central1
Description:
Google App Engine and Cloud Functions experienced elevated request latencies for a duration of 49 minutes. From preliminary analysis, the root cause of the issue is an unexpected surge in traffic which increased resource usage. The issue was mitigated by implementing safeguards to prevent resource starvation.
We continue to investigate the nature of the traffic surge and we will identify additional safeguards to prevent recurrence of the issue.
Customer Impact:
Affected customers would have experienced elevated latencies for App Engine app serving and Cloud Function executions.
Less than 1% of the customers in us-central1 would have seen App Engine and Cloud Function requests failing with errors.
04:03 PM
The issue with Google App Engine has been resolved for all affected users as of Tuesday, 2022-05-24 07:50 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
03:51 PM
Summary: Degraded serving for Google App Engine and Cloud Functions users in us-central1
Description: We are experiencing an issue with Google App Engine beginning at Tuesday, 2022-05-24 07:01 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-05-24 08:15 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers are seeing a high serving latency and request failures
Workaround: None at this time