Tensorflow: How to convert NaNs to a number?

Question

I'm trying to calculate the entropy of the weights while training my graph and to use it for regularization. This of course involves w*tf.log(w), and as my weights are changing some of them are bound to get into a region which results in NaNs being returned.

Ideally I would include a line in my graph setup:

w[tf.is_nan(w)] = <number>

but tensorflow doesn't support assigning like that. I could of course create an operation, but that wouldn't work because I need for it to happen during the execution of the entire graph. I can't wait for the graph to execute and then 'fix' my weights, is has to be part of the graph execution.

I haven't been able to find an equivalent to np.nan_to_num in the docs.

Anybody have an idea?

(For obvious reasons, adding an epsilon doesn't work)

Answer 1

I think you need to use tf.select.

w = tf.select(tf.is_nan(w), tf.ones_like(w) * NUMBER, w); #if w is nan use 1 * NUMBER else use element in w

---

Update: TensorFlow 1.0 has deprecated tf.select in favor of Numpy compatible tf.where.