Google Cloud MINOR
We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.
December 28, 2022 · 02:04 AM UTC – 12:02 PM UTC · Duration: 9h 58min
Affected Services
Vertex AI AutoML ImageVertex AI Online PredictionVertex AI Batch PredictionCloud Machine Learning
Timeline
12:02 PM
The issue with Vertex AI AutoML Image, Vertex AI Batch Prediction, Vertex AI Online Prediction has been resolved for all affected projects as of Wednesday, 2022-12-28 04:02 US/Pacific.
We thank you for your patience while we worked on resolving the issue.
07:27 AM
Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.
Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Wednesday, 2022-12-28 10:00 US/Pacific with current details.
Diagnosis: Users might experience higher latency.
Workaround: None.
07:05 AM
Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.
Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-27 23:30 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Users might experience higher latency.
Workaround: None.
03:59 AM
Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.
Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-27 23:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Users might experience higher latency.
Workaround: None.
02:04 AM
Summary: We are experiencing increased tail latency for predictions on Vertex AI Online Prediction.
Description: We are experiencing an issue with Vertex AI Online Prediction, on beginning at Wednesday, 2022-12-21 19:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2022-12-27 20:00 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Users might experience higher latency.
Workaround: None.