Menu
If I have a fairly complex equation for the cost like this:
cost = tf.reduce_sum( tf.multiply( y , tf.log(y/abs(yy))) + \
tf.multiply( (1 - y) , tf.log((1-y)/abs(1-yy)) ) )
will tensorflow still figure out the correct backpropagation equations for this? In other words, how can I be sure that the gradients are calculated correctly?
910
Gradient tapesTensorFlow "records" relevant operations executed inside the context of a tf. GradientTape onto a "tape". TensorFlow then uses that tape to compute the gradients of a "recorded" computation using reverse mode differentiation.
TensorFlow has built in function to create tensors for use in variables. For example, we can create a zero filled tensor of predefined shape using the tf. zeros() function as follows. We can evaluate tensors with calling a run() method on our session.
As described in this anwser, __add__ op (and __mul__ as well btw) are overloaded, therefore
cost = tf.reduce_sum(tf.add(
tf.multiply( y , tf.log(y/abs(yy))),
tf.multiply((1 - y) , tf.log((1-y)/abs(1-yy)))
))
is equivalent to
cost = tf.reduce_sum(y * tf.log(y/abs(yy)) + (1 - y) * tf.log((1-y)/abs(1-yy)))
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
