Converting TensorFlow tensor into Numpy array

Question

Problem Description

I am trying to write a custom loss function in TensorFlow 2.3.0. To calculate the loss, I need the y_pred parameter to be converted to a numpy array. However, I can't find a way to convert it from <class 'tensorflow.python.framework.ops.Tensor'> to numpy array, even though there seem to TensorFlow functions to do so.

Code Example

def custom_loss(y_true, y_pred):
    print(type(y_pred))
    npa = y_pred.make_ndarray()
    ...
    

if __name__ == '__main__':
    ...
    model.compile(loss=custom_loss, optimizer="adam")
    model.fit(x=train_data, y=train_data, epochs=10)

gives the error message: AttributeError: 'Tensor' object has no attribute 'make_ndarray after printing the type of the y_pred parameter: <class 'tensorflow.python.framework.ops.Tensor'>

What I have tried so far

Looking for a solution I found this seems to be a common issue and there a couple of suggestions, but they did not work for me so far:

1. " ... so just call .numpy() on the Tensor object.": How can I convert a tensor into a numpy array in TensorFlow?

so I tried:

def custom_loss(y_true, y_pred):
    npa = y_pred.numpy()
    ...

giving me AttributeError: 'Tensor' object has no attribute 'numpy'

2. "Use tensorflow.Tensor.eval() to convert a tensor to an array": How to convert a TensorFlow tensor to a NumPy array in Python

so I tried:

def custom_loss(y_true, y_pred):
    npa = y_pred.eval(session=tf.compat.v1.Session())
    ...

giving me one of the longest trace of error messages I ever have seen with the core being:

InvalidArgumentError: 2 root error(s) found.
      (0) Invalid argument: You must feed a value for placeholder tensor 'functional_1/conv2d_2/BiasAdd/ReadVariableOp/resource' with dtype resource
         [[node functional_1/conv2d_2/BiasAdd/ReadVariableOp/resource (defined at main.py:303) ]]
         [[functional_1/cropping2d/strided_slice/_1]]
      (1) Invalid argument: You must feed a value for placeholder tensor 'functional_1/conv2d_2/BiasAdd/ReadVariableOp/resource' with dtype resource
         [[node functional_1/conv2d_2/BiasAdd/ReadVariableOp/resource (defined at main.py:303) ]]

also having to call TensorFlow Compatibility Functions from Version 1.x does not feel very future-proof, so I do not like this approach too much anyhow.

3. Looking at the TensorFlow Docs there seemed to be the function I needed just waiting: tf.make_ndarray Create a numpy ndarray from a tensor.

so I tried:

def custom_loss(y_true, y_pred):
    npa = tf.make_ndarray(y_pred)
    ...

giving me AttributeError: 'Tensor' object has no attribute 'tensor_shape'

Looking at the example in the TF documentation they use this on a proto_tensor, so I tried converting to a proto first:

def custom_loss(y_true, y_pred):
    proto_tensor = tf.make_tensor_proto(y_pred)
    npa = tf.make_ndarray(proto_tensor)
    ...

but already the tf.make_tensor_proto(y_pred) raises the error: TypeError: Expected any non-tensor type, got a tensor instead.

Also trying to make a const tensor first gives the same error:

def custom_loss(y_true, y_pred):
    a = tf.constant(y_pred)
    proto_tensor = tf.make_tensor_proto(a)
    npa = tf.make_ndarray(proto_tensor)
    ...

There are many more posts around this but it seems they are all coming back to these three basic ideas. Looking forward to your suggestions!

Answer 1

y_pred.numpy() works in TF 2 but AttributeError: 'Tensor' object has no attribute 'make_ndarray indicates that there are parts of your code that you are not running in Eager mode as you would otherwise not have a Tensor object but an EagerTensor.

To enable Eager Mode, put this at the beginning of your code before anything in the graph is built:

tf.config.experimental_run_functions_eagerly(True)

Second, when you compile your model, add this parameter:

model.compile(..., run_eagerly=True, ...)

Now you're executing in Eager Mode and all variables actually hold values that you can both print and work with. Be aware that switching to Eager mode might require additional adjustments to your code (see here for an overview).