How should I handle input data with nan values in TensorFlow?

Question

I have a multilayer perceptron with a sigmoid loss (tf.nn.sigmoid_cross_entropy_with_logits) and an Adam optimizer (tf.train.AdamOptimizer). My input data has several features and some nan feature-values. When I replace the nan values with 0, I get a result, however, when I do not replace the nan values, I get loss=nan.

What is the best way to handle nan values in TensorFlow, and how can I use my input data with nan values without replacing them with 0?

Answer 1

Question

How can I somehow tell my network to ignore some input data. For example when the input data is nan

Answer

This is very similar to adding a mask to your input data. You want your input data to pass through, nans turned to zeros, but you want somehow to also signal to the neural network to ignore where the nans were and pay attention to everything else.

In this question about adding a mask I review how a mask can successfully be added to an image but also give a code demonstration for a non-image problem.

• First create a mask, 1's where data exists in the input and 0's where nan exist.
• Second, clean up the input converting nans to 0's, or 0.5's, or anything really.
• Third, stack the mask onto the input. If the input is an image, then the mask becomes another colour channel.

The code in the masking question shows that when the mask is added the neural net is able to learn well and when the mask is not added it is not able to learn well.