fastai learner requirements and batch prediction

Question

I previously trained a resnet34 model using the fastai library, and have the weights.h5 file saved. With the latest version of fastai, do I still need to have non-empty train and valid folders in order to import my learner and predict on the test set?

Also, I’m currently looping through every test image and using learn.predict_array, but is there a way to predict in batches on a test folder?

Example of what I’m currently doing just to load/predict:

PATH = '/path/to/model/'
sz = 224
arch=resnet34
tfms = tfms_from_model(resnet34, sz, aug_tfms=transforms_side_on, max_zoom=1.1)
data = ImageClassifierData.from_paths(PATH, tfms=tfms, bs=64)
learn = ConvLearner.pretrained(arch, data, precompute=False)
learn.unfreeze()
learn.load('224_all')

imgs = sorted(glob(os.path.join(test_path, '*.jpg')))
preds = []
_,val_tfms = tfms_from_model(resnet34, 224)
for n, i in enumerate(imgs):
        im = val_tfms(open_image(i))[None]
        preds.append(1-np.argmax(learn.predict_array(im)[0]))

There must be a cleaner way to do this by now, no?

Answer 1

In fastai, you can now export and load a learner to do prediction on the test set without having to load a non empty training and validation set. To do that, you should use export method and load_learner function (both are defined in basic_train).

In your current situation, you might have to load your learner the old way (with a train/valid dataset), then export it and you'll be able to use load_learner to do your predictions on your test set.

I'll leave a link to the documentation :

-https://docs.fast.ai/basic_train.html#Deploying-your-model

This should clarify any follow up questions.

Answer 2

data = ImageClassifierData.from_paths(PATH, tfms=tfms, bs=64)
learn = ConvLearner.pretrained(arch, data, precompute=False)
learn.unfreeze()
learn.load('224_all')

preds = learn.predict(is_test=True)