TensorFlow: cast a float64 tensor to float32

Question

I am trying to use: train = optimizer.minimize(loss) but the standard optimizers do not work with tf.float64. Therefore I want to truncate my loss from tf.float64 to only tf.float32.

Traceback (most recent call last):
  File "q4.py", line 85, in <module>
    train = optimizer.minimize(loss)
  File "/Library/Python/2.7/site-packages/tensorflow/python/training/optimizer.py", line 190, in minimize
    colocate_gradients_with_ops=colocate_gradients_with_ops)
  File "/Library/Python/2.7/site-packages/tensorflow/python/training/optimizer.py", line 229, in compute_gradients
    self._assert_valid_dtypes([loss])
  File "/Library/Python/2.7/site-packages/tensorflow/python/training/optimizer.py", line 354, in _assert_valid_dtypes
    dtype, t.name, [v for v in valid_dtypes]))
ValueError: Invalid type tf.float64 for Add_1:0, expected: [tf.float32].

Answer 1

The short answer is that you can convert a tensor from tf.float64 to tf.float32 using the tf.cast() op:

loss = tf.cast(loss, tf.float32)

The longer answer is that this will not solve all of your problems with the optimizers. (The lack of support for tf.float64 is a known issue.) The optimizers require that all of the tf.Variable objects that you are trying to optimize must also have type tf.float32.