WARNING:tensorflow:11 out of the last 11 calls to triggered tf.function retracing

Question

Anyone know the reason for this error?

WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.
WARNING:tensorflow:11 out of the last 11 calls to <function Model.make_predict_function.<locals>.predict_function at 0x000001F9D1C05EE0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:11 out of the last 11 calls to <function Model.make_predict_function.<locals>.predict_function at 0x000001F9D5604670> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
C:\Users\User\anaconda3\lib\site-packages\sklearn\cluster\_kmeans.py:973: FutureWarning: 'n_jobs' was deprecated in version 0.23 and will be removed in 0.25.
  warnings.warn("'n_jobs' was deprecated in version 0.23 and will be"

Answer 1

{TLDR} try replacing model.predict(x) by model(x)

My Solution

I also had issues with the warning:

WARNING:tensorflow:11 out of the last 11 calls to <function Model.make_predict_function.<locals>.predict_function at 0x000001F9D1C05EE0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for  more details.

I was able to solve it with replacing model.predict(x) by directly using model(x)

---

Background Infos where I encountered the problem

I am predicting time series and fit the last layer of my model every new sampling time to the latest data. Therefore I

1. generate and fit a base model and freeze all layers + put a top layer
2. fit to new data and predict inside a loop

I tried to implement a custom predict function with a signature as suggested from the Warning and @TFer2. This however yielded the error

RuntimeError: Detected a call to `Model.predict` inside a `tf.function`. `Model.predict is a high-level endpoint that manages its own `tf.function`. Please move the call to `Model.predict` outside of all enclosing `tf.function`s. Note that you can call a `Model` directly on `Tensor`s inside a `tf.function` like: `model(x)`.

With this error I was then able to solve the issue.

Answer 2

If you call a function with the same argument type tensorflow will reuse a previously traced graph else will create new graph.

A function determines, whether to reuse a traced concrete function by computing a cache key from an input's args and kwargs:

• A key generated for a tf.Tensor argument is its shape and type (input signatures)
• The key generated for a tf.Variable argument is its id().
• A key generated for a python primitive is its value.
• The key generated for nested dicts, lists, tuples, namedtuples, and attrs is the flattened tuple.

Retracing ensures that tensorflow generates correct graphs for each set of inputs. But it is expensive.

You have to avoid execessive retracing else tensorflow will usually issue a warning as above.

There are few ways that you can control tracing behavior:

• Specify an input_signature in tf.function
• specify a [None] dimension in tf.TensorSpec to allow for flexibility in trace reuse
• Cast python arguments to Tensors to reduce retracing

For more details you can refer Better performance with tf.function.