How to prevent division by zero or replace infinite values in Theano?

Question

I'm using a cost function in Theano which involves a regularizer term that requires me to compute this term:

T.sum(c / self.squared_euclidean_distances)

as some values of self.squared_euclidean_distances might be zero this produces Nan values. How can i work around this problem? I tried to use T.isinf but were not successful. One solution would be to remove zeros in self.squared_euclidean_distances into a small number or replace infinite numbers in T.sum(c / self.squared_euclidean_distances) to zero. I just don't know how to replace those values in Theano.

Answer 1

Take a look at T.switch. You could do for example

T.switch(T.eq(self.squared_euclidean_distances, 0), 0, c / self.squared_euclidean_distances)

(Or, upstream, you make sure that you never compare a vector with itself using squared euclidean distance.)