Hey, I had a question regarding Vertex AI batch predictions. It would be great if someone can help with.
If in Vertex Ai batch prediction request let's say we have 15-20 bad inputs. And as per the process we know the batch prediction job divides the whole request into mini batches(default size 64) and uses Data flow to send batch requests to the endpoint provisioned and managed by GCP under the hood .
1. So if a particular mini batch has lets say 5 bad inputs will the whole mini batch fail after 3 retries(retries are fixed what I read from documentation). Because what i think is the whole mini batch is fed into model.predict() so even if a single a instance is corrupted the whole mini batch fails?
2. If a single/multiple mini batch fails will the whole job fail? basically will the final batch prediction job status would be succeeded or failed?
LinkedIn
Twitter
