Google Cloud MINOR

New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures

April 21, 2023 · 01:06 AM UTC – 05:18 AM UTC · Duration: 76h 12min

Affected Services

Dataproc MetastoreGoogle Cloud Dataproc

Timeline

05:18 AM
The issue with Dataproc Metastore has been resolved for all affected users as of Sunday, 2023-04-23 16:13 US/Pacific. We thank you for your patience while we worked on resolving the issue.
08:10 PM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team has validated the fix and have a mitigation rollout in progress. The mitigation is expected to complete by Monday, 2023-04-24 16:00 US/Pacific. We will provide more information by Monday, 2023-04-24 17:00 US/Pacific. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time.
04:52 PM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team has determined that further investigation is required to mitigate the issue. We will provide an update by Saturday, 2023-04-22 12:00 US/Pacific with current details. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Some customers may experience issues with GKE clusters autoscaling the nodepool on N1 standard machine types. Workaround: None at this time.
07:40 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Friday, 2023-04-21 09:00 US/Pacific with current details. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time.
05:34 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Friday, 2023-04-21 00:30 US/Pacific with current details. Diagnosis: Newly created (Dataproc Metastore) DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time.
05:32 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team is continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Friday, 2023-04-21 00:30 US/Pacific with current details. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time.
04:08 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team is continues the testing and validation of the mitigation steps to reduce the customer impact. We will provide an update by Thursday, 2023-04-20 22:00 US/Pacific with current details. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Existing service updates will continue to work except for updates to ports, which may fail with a 500 internal error. Workaround: None at this time.
02:17 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team is currently testing the mitigation steps to reduce the customer impact. We will provide an update by Thursday, 2023-04-20 20:00 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Workaround: None at this time.
01:46 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2023-04-20 18:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing 500 internal error during creation. Workaround: None at this time.
01:29 AM
Summary: New DPMS service created in us-east4, europe-west3 and us-west2 are experiencing failures Description: We are experiencing an issue with Dataproc Metastore. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2023-04-20 18:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Newly created DPMS service in us-east4, europe-west3 and us-west2 are experiencing time outs during creation. Workaround: None at this time.