Google Cloud MAJOR
Cloud logs might be unavailable for 25% of customers in global location.
October 27, 2021 · 03:58 AM UTC – 07:07 AM UTC · Duration: 3h 9min
Affected Services
OperationsCloud Logging
Timeline
07:26 PM
We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support
(All Times US/Pacific)
Incident Start: 26 October 2021 19:58
Incident End: 26 October 2021 23:07
Duration: 3 hours, 9 minutes
Affected Services and Features:
Google Cloud Logging - Retrieval
Regions/Zones: Global
Description:
Some Google Cloud Logging customers experienced unavailable logs for a duration of 3 hours and 9 minutes. From preliminary analysis, the root cause of the issue was an underlying file system upgrade which required an expensive re-initialization operation.
Customer Impact:
25% of projects were unable to query their logging data and queries would either time out or return a continuation token.
05:50 AM
The issue with Cloud Logging is believed to be affecting a very small number of customers and our Engineering Team is working on it.
If you have questions or are impacted, please open a case with the Support Team and we will work with you until this issue is resolved.
We thank you for your patience while we're working on resolving the issue.
05:49 AM
Summary: Cloud logs might be unavailable for 25% of customers in global location
Description: Mitigation work is currently underway by our engineering team.
The mitigation is expected to complete by Tuesday, 2021-10-26 22:40 US/Pacific.
We will provide an update by Tuesday, 2021-10-26 22:52 US/Pacific with current details.
Diagnosis: Logs won't be available, high latency when reading data.
Workaround: Unknown
05:14 AM
Summary: Cloud logs might be unavailable for 25% of customers in global location
Description: We are experiencing an issue with Cloud Logging beginning at Tuesday, 2021-10-26 19:57 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2021-10-26 21:52 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Logs won't be available, high latency when reading data.
Workaround: Unknown