us-west1: Cloud Logging experiencing elevated error rate while reading logs

We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using https://cloud.google.com/support (All Times US/Pacific) Incident Start: 18 October 2021 07:30 Incident End: 20 October 2021 04:35 Duration: 1 day, 21 hours, 5 minutes Affected Services and Features: Google Cloud Logging Regions/Zones: us-west1 Description: Google Cloud Logging experienced periods of increased latency and issues loading logs for 45 hours and 5 minutes in us-west1 due to resource contention related to unexpected increases in traffic. Our engineering team scaled up resources to mitigate the issue. Engineering team subsequently identified the source of the increase in traffic, and confirmed that it had normalized. Our engineering team has identified the following timeframes where customers may have experienced significant impact: First Impact Timeframe Start: 18 October 2021 09:45 End: 18 October 2021 23:50 Second Impact Timeframe Start: 19 October 2021 07:44 End: 19 October 2021 18:50 Customer Impact: Customers may have experienced increased latency in Cloud Console and issues loading logs.

The issue with Cloud Logging has been resolved for all affected users as of Monday, 2021-10-18 13:18 US/Pacific. We thank you for your patience while we worked on resolving the issue.

Summary: us-west1: Cloud Logging experiencing elevated error rate while reading logs Description: We are experiencing an issue with Cloud Logging beginning at Monday, 2021-10-18 07:27 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Monday, 2021-10-18 13:50 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Increased latency in Cloud Console and issues with loading logs Workaround: None at this time.

Summary: us-west1: Cloud Logging experiencing elevated error rate while reading logs Description: We are experiencing an issue with Cloud Logging beginning at Monday, 2021-10-18 07:27 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Monday, 2021-10-18 12:50 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Increased latency in Cloud Console and issues with loading logs Workaround: None at this time.

Summary: us-west1: Cloud Logging experiencing elevated error rate while reading logs Description: We are experiencing an issue with Cloud Logging beginning at Monday, 2021-10-18 07:27 US/Pacific. Our engineering team continues to investigate the issue. We will provide an update by Monday, 2021-10-18 11:50 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Increased latency in Cloud Console and issues with loading logs Workaround: None at this time.