Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to see estimated time for training a model in Google AutoML Vision?

I'm attempting to train a model to identify certain tags in images. I tried using the 1 hour free version and after an hour the training ended. The results weren't as accurate as I would like, so I took the plunge and selected the option which did not define a specific time limit for training the model.

At this time, it is showing "training vision classification model" and "Training can take 15 minutes to several hours or more, depending on the compute hours assigned. In the meantime, you can close this window. You will be emailed once training completes."

It's been running for over 24 hours now. Google charges $20 per training hour so I would like to know if there is a way to see how much longer it would take to fully train the model.

I'm using it using 2594 images with only single labels.

I understand Google AutoML Vision is still in beta phase but at this time, there is no way to know how long this training will last. I'm under a strict budget so I will stop the training if it exceeds the time -- but I would still like to know if it was near completion.

like image 510
Jay Song Avatar asked Jan 04 '19 05:01

Jay Song


People also ask

Can you train Google vision?

Open the AutoML Vision Object Detection UI. The Datasets page shows the available datasets for the current project. Select the dataset you want to use to train the custom model. When the dataset is ready, select the Train tab and Train new model button.

What is the maximum image size required to make an inference in AutoML?

1.5MB maximum size. 1024 pixels by 1024 pixels suggested maximum. For images much larger than 1024 pixels by 1024 pixels some image quality may be lost during AutoML Vision Object Detection's image normalization process.

What algorithm does Google AutoML use?

Google open sources AutoML model search algorithm Techniques like neural architecture search (NAS), use algorithms, like reinforcement learning (RL), evolutionary algorithms, and combinatorial search, to build a neural network out of a given search space.


1 Answers

The time required to train your model will completely depends and vary based on the amount of data included within your dataset and the accuracy desired for your model; therefore, I think it is not possible to determine the number of hours needed to complete the training phase. The GCP pricing documentation mentions the following suggestion:

Many customers find that one hour is sufficient to build an experimental model and use additional training hours to increase accuracy to a production level.

Note: Keep in mind that the accuracy of your model generally depends on how long you allow it to train and the quality of your training dataset.

On the other hand, in case you have an specific budget for your model, it is recommended to use the training budget parameter in order to specify the number of hours of training to use; In this way, AutoML Vision can guarantee that the actual training time will be less than or equal to the training budget. I suggest you to take a look on the AutoML Vision API Tutorial where you can find an example of the usage of the training model property.

Additionally, it is important to note that, when training a model, a compute hour represents internal compute usage, which means that this time might not exactly match an actual hour on the clock, as mentioned in the Training costs. Based on this, I recommend you to consider this aspect when calculating the your budget in order to avoid exceeding it.

like image 169
Armin_SC Avatar answered Sep 17 '22 14:09

Armin_SC