Google Cloud MAJOR
We are experiencing an issue with Chronicle Security Dashboards
March 22, 2024 · 04:09 PM UTC – 06:29 PM UTC · Duration: 2h 20min
Affected Services
Chronicle Security
Timeline
10:39 PM
Mini Incident Report
We apologize for the inconvenience this service disruption may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support
(All Times US/Pacific)
Incident Start: 22 March 2024 08:09
Incident End: 22 March 2024 10:29
Duration: 2 hours, 20 minutes
Affected Services and Features:
Google Cloud Platform - Chronicle SIEM
Regions/Zones: Multi-regions: us, eu
Description:
Chronicle SIEM experienced an issue with Chronicle Security Dashboards for a duration of 2 hours, 20 minutes. From preliminary analysis, the root cause of the issue was related to a recent product change that was deployed to production. Google Engineers rolled back the change which mitigated the issue at 10:29 US/Pacific
Customer Impact:
The Chronicle Security Dashboards failed to load.
100% failure rate for related API calls.
Affected customers would have experienced an error when accessing dashboards: "An error occurred while loading dashboards. Please try refreshing the page in a few moments."
06:39 PM
The issue with Chronicle Security dashboards has been resolved for all affected users as of Friday, 2024-03-22 10:29 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
06:16 PM
Summary: We are experiencing an issue with Chronicle Security Dashboards
Description: Our engineering team has identified the root cause and have initiated a rollback.
The mitigation is expected to complete by Friday, 2024-03-22 11:30 US/Pacific.
We will provide more information by Friday, 2024-03-22 12:00 US/Pacific.
Diagnosis: Customers will receive the following error:
"An error occurred while loading dashboards. Please try refreshing the page in a few moments."
Workaround: None at this time.
05:57 PM
Summary: We are experiencing an issue with Chronicle Security Dashboards
Description: We are experiencing an issue with Chronicle Security dashboard beginning at Friday, 2024-03-22 08:09 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Friday, 2024-03-22 10:30 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers will receive the following error:
"An error occurred while loading dashboards. Please try refreshing the page in a few moments."
Workaround: None at this time.