Google Cloud MINOR

Vertex AI Search: Delays in Retail Search model training

January 11, 2024 · 06:26 PM UTC – 11:19 PM UTC · Duration: 4h 53min

Affected Services

Recommendation AICloud Machine Learning

Timeline

11:19 PM
The issue with Recommendation AI, Vertex AI Search has been resolved as of Thursday, 2024-01-11 14:50 US/Pacific. We understand that this issue had impacted your ability to access and use our services, we deeply appreciate your patience and cooperation while we worked to mitigate the issue. Thank you for choosing us.
10:06 PM
Summary: Vertex AI Search: Retail Search model training delayed Description: Engineers have partially mitigated the issue. At this time Retail Search model training has resumed for new events. Existing, unprocessed events are also processing, however this is likely to take some time to complete. We do not have an ETA for full resolution at this point. We will provide an update by Friday, 2024-01-12 13:00 US/Pacific with current details. Diagnosis: Retail Search model training may be delayed or stopped. Workaround: None at this time.
09:50 PM
Summary: Vertex AI Search: Retail Search model training delayed Description: Mitigation work remains underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2024-01-11 17:00 US/Pacific. Diagnosis: Retail Search model training may be delayed or stopped. Workaround: None at this time.
08:06 PM
Summary: Vertex AI Search: Retail Search model training delayed Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2024-01-11 14:00 US/Pacific. Diagnosis: Retail Search model training may be delayed or stopped. Workaround: None at this time.
07:07 PM
Summary: Vertex AI Search: Retail Search model training delayed Description: Mitigation work is currently underway by our engineering team. We do not have an ETA for mitigation at this point. We will provide more information by Thursday, 2024-01-11 12:15 US/Pacific. Diagnosis: Retail Search model training may be delayed or stopped. Workaround: None at this time.
06:48 PM
Summary: Vertex AI Search: Retail Search model training delayed Description: We are experiencing an issue with Recommendation AI, Vertex AI Search. Our engineering team continues to investigate the issue. We will provide an update by Thursday, 2024-01-11 12:30 US/Pacific with current details. We apologize to all who are affected by the disruption. Diagnosis: Retail Search model training may be delayed or stopped. Workaround: None at this time.