Keras: difference between test_on_batch and predict_on_batch

Question

In Philippe Remy's blog post on stateful LSTM, at the bottom he says "You may have to do your validation/testing manually by calling predict_on_batch() or test_on_batch()".

Looking at the documentation, predict_on_batch does this:

predict_on_batch(self, x)

Returns predictions for a single batch of samples.

Arguments

    x: Input samples, as a Numpy array.

Returns

Numpy array(s) of predictions.

And test_on_batch does this:

test_on_batch(self, x, y, sample_weight=None)

Test the model on a single batch of samples.

Arguments

    x: Numpy array of test data, or list of Numpy arrays if the model has multiple inputs. If all inputs in the model are named, you can also pass a dictionary mapping input names to Numpy arrays.
    y: Numpy array of target data, or list of Numpy arrays if the model has multiple outputs. If all outputs in the model are named, you can also pass a dictionary mapping output names to Numpy arrays.
    sample_weight: Optional array of the same length as x, containing weights to apply to the model's loss for each sample. In the case of temporal data, you can pass a 2D array with shape (samples, sequence_length), to apply a different weight to every timestep of every sample. In this case you should make sure to specify sample_weight_mode="temporal" in compile().

Returns

Scalar test loss (if the model has a single output and no metrics) or list of scalars (if the model has multiple outputs and/or metrics). The attribute model.metrics_names will give you the display labels for the scalar outputs

I'm using train_on_batch in my training loop, basically exactly as the blog post does. How do I know whether to use predict_on_batch or test_on_batch for cross-validation?

Answer 1

So basically:

• predict_on_batch is providing you a set of predictions for data you feed as an argument. If you train a network for recognizing cats and dogs - once you feed an image to predict_on_batch method - you'll get probability if a given image has a cat or a dog in it. You can also use these probabilities to evaluate your model.
• test_on_batch is providing you a scalar(s) which measures how well is your model performing. Given images and true labels - it's computing a bunch of losses and metrics (defined at model compilation) which measure how your model fits the data.

The suffix on_batch comes from the fact that model is performing all computations at once. Sometimes it's not feasible so it's better to divide data in small chunks and perform multiple computations. keras provides you functions to do this automatically. So predict is equivalent to predict_on_batch and evaluate - to test_on_batch.

In order to choose function used during cross-validation - you need to use:

• predict / predict_on_batch when you need the actual predictions, not only metrics values. In this case, however, you still need to compute metrics values.
• test_on_batch / evaluate when you simply need metrics values.

So - as you may see - you can actually combine these two functions while evaluating your model. If you simply want metrics - try test_on_batch / evaluate as it may save you a lot of computations.