Google Cloud MINOR
Multiple Google Cloud Platform services impacted globally with operational latency
June 13, 2023 · 01:15 AM UTC – 05:48 AM UTC · Duration: 4h 33min
Affected Services
ApigeeGoogle Compute EngineCloud MemorystoreGoogle Cloud DataflowGoogle Cloud NetworkingGoogle Cloud ComposerGoogle Cloud SQLCloud Load BalancingCloud FilestoreGoogle App EngineAlloyDB for PostgreSQLVirtual Private Cloud (VPC)Memorystore for MemcachedMemorystore for Redis
Timeline
06:44 AM
The issue with Apigee, Cloud Filestore, Cloud Load Balancing, Cloud Memorystore, Google App Engine, Google Cloud Composer, Google Cloud Dataflow, Google Cloud Networking, Google Cloud SQL, Google Compute Engine, Google Kubernetes Engine, Virtual Private Cloud and AlloyDB for PostgreSQL has been resolved for all affected users as of Monday, 2023-06-12 21:48 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
06:07 AM
Summary: Multiple Google Cloud Platform services impacted globally with operational latency
Description: Mitigation work is still underway by our engineering team.
We will provide more information by Monday, 2023-06-12 22:45 US/Pacific.
Diagnosis: Users may observe failures or delayed operations for affected GCP services.
Google Compute Engine
Impact/Diagnosis: Impacted users may experience elevated latency for API calls to global, regional and zonal resources.
Cloud Memorystore
Impact/Diagnosis: Users may not be able to create new Redis instances, and likely not be able to delete instances.
Google Cloud Composer
Impact/Diagnosis: Users may observe failures while running create operations using Cloud Composer.
Google Cloud Dataflow
Impact/Diagnosis: Impacted customers may see increased latency during WorkerPool start up and for Dataflow jobs.
Google Cloud App Engine
Impact/Diagnosis: Flex deployments are failing or timing out
Cloud Filestore
Impact/Diagnosis: Attempts to create, delete, or back up instances may fail.
Google Cloud SQL
Impact/Diagnosis: Instance creation, backup create/restore/delete operations are failing with error 503: "The service is currently unavailable."
Apigee X
Impact/Diagnosis: Attempts to create, delete, or back up instances may fail.
Cloud Load Balancing
Impacted customers may experience failures while performing create, delete, or edit operations on Layer 7 internal or external load balancers or their dependencies.
Workaround: None at this time for all services except Google Cloud SQL where retrying the failed operation may help.
05:46 AM
Summary: Multiple Google Cloud Platform services impacted globally with operational latency
Description: We are experiencing an issue with Cloud Memorystore, Google Compute Engine, Google Cloud Composer, Google Cloud Networking, Cloud Filestore, Google App Engine, Apigee, Google Cloud SQL, Google Cloud Dataflow beginning at Monday, 2023-06-12 17:15 US/Pacific.
Our engineering team continues to investigate the issue. We will provide an update by Monday, 2023-06-12 22:30 US/Pacific with current details. We apologize to all who are affected by the disruption.
Diagnosis: Users may observe failures or delayed operations for affected GCP services.
Google Compute Engine
Impact/Diagnosis: Impacted users may experience elevated latency for API calls to global, regional and zonal resources.
Cloud Memorystore
Impact/Diagnosis: Users may not be able to create new Redis instances, and likely not be able to delete instances.
Google Cloud Composer
Impact/Diagnosis: Users may observe failures while running create operations using Cloud Composer.
Google Cloud Dataflow
Impact/Diagnosis: Impacted customers may see increased latency during WorkerPool start up and for Dataflow jobs.
Google Cloud App Engine
Impact/Diagnosis: Flex deployments are failing or timing out
Cloud Filestore
Impact/Diagnosis: Attempts to create, delete, or back up instances may fail.
Google Cloud SQL
Impact/Diagnosis: Instance creation, backup create/restore/delete operations are failing with error 503: "The service is currently unavailable."
Apigee X
Impact/Diagnosis: Attempts to create, delete, or back up instances may fail.
Workaround: None at this time.
05:23 AM
Summary: Multiple Google Cloud Platform services impacted globally with operational latency
Description: We are experiencing an issue with Cloud Memorystore, Google Compute Engine, Google Cloud Composer, Google Cloud Networking, Cloud Filestore, Google App Engine, Google Kubernetes Engine, Apigee, Google Cloud SQL, Google Cloud Dataflow beginning at Monday, 2023-06-12 17:15 US/Pacific.
Our engineering team continues to investigate the issue. We will provide an update by Monday, 2023-06-12 22:00 US/Pacific with current details. We apologize to all who are affected by the disruption.
Diagnosis: Users may observe failures or delayed operations for affected GCP services.
Workaround: None at this time.
04:48 AM
Summary: Multiple Google Cloud Platform services impacted globally with operational latency
Description: We are experiencing an issue with Cloud Memorystore, Google Compute Engine, Google Cloud Composer, Google Cloud Networking, Cloud Filestore, Google App Engine, Google Kubernetes Engine, Apigee, Google Cloud SQL, Google Cloud Dataflow beginning at Monday, 2023-06-12 17:15 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Monday, 2023-06-12 21:30 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Users may observe failures or delayed operations for affected GCP services.
Workaround: None at this time.