Google Cloud MINOR

Customers using AutoML translations may experience high latency and 'resource exhausted' failures.

August 2, 2024 · 05:30 PM UTC – 01:36 AM UTC · Duration: 8h 6min

Affected Services

AutoML TranslationCloud Machine Learning

Timeline

01:36 AM
Our Engineering team has successfully concluded validations and confirmed that issue has been mitigated. If you have questions or require further assistance, please open a case with the Support Team and we will work with you to ensure that your concerns are addressed. We thank you for your patience while we worked on resolving the issue.
10:58 PM
Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We believe the issue with AutoML Translation is mitigated as per our monitoring. Our engineering team is in the process of validating the mitigation further, to ensure the issue is fully resolved for all our customers. We will provide an update by Friday, 2024-08-02 19:00 US/Pacific with current details. Diagnosis: When users use translation with custom models, most requests will either fail with timeout/resource exhausted, or have latency up to 2 mins. Workaround: None at this time.
07:53 PM
Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We believe the issue with AutoML Translation is mitigated as per our monitoring. Our engineering team is still validating further to ensure the issue is fully resolved for all our customers. We will provide an update by Friday, 2024-08-02 15:00 US/Pacific with current details. Diagnosis: When users use translation with custom models, most requests will either fail with timeout/resource exhausted, or have latency up to 2 mins. Workaround: None at this time.
06:18 PM
Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We believe the issue with AutoML Translation is mitigated as per our monitoring. Our engineering team is currently validating further to ensure the issue is fully resolved for all our customers. We will provide an update by Friday, 2024-08-02 12:00 US/Pacific with current details. Diagnosis: When users use translation with custom models, most requests will either fail with timeout/resource exhausted, or have latency up to 2 mins. Workaround: None at this time.
05:56 PM
Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We are experiencing an issue with AutoML translations Our engineering team continues to investigate the issue. We will provide an update by Friday, 2024-08-02 11:08 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: AutoML translations may experience high latency and resource exhausted failures. Workaround: None at this time.
05:49 PM
Summary: Customers using AutoML translations may experience high latency and resource exhausted failures. Description: We are experiencing an issue with Cloud Translation. Our engineering team continues to investigate the issue. We will provide an update by Friday, 2024-08-02 11:08 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: None at this time. Workaround: None at this time.