tf.floor return None gradient
it means the weights before floor operation won't be update, right?
but I still need the gradient to update weights
Isn't it weird? sometimes we use floor, ceil... but they can't deliver gradient or this is right action
as well as tf.cast, the gradient return none
Here is an issue #897 to discuss the gradient, but I don't understand why return None is good
numa = tf.constant(5.3)
res1 = numa * 2
res2 = tf.floor(numa *2)
print tf.gradients(res1, numa)
# <type 'list'>: [<tf.Tensor 'gradients/mul_grad/Reshape:0' shape=() dtype=float32>]
print tf.gradients(res2, numa)
# None
Can I modify math_grad.py directly? need other action?
something like delete math_grad.pyc?
@ops.RegisterGradient("Floor")
def _FloorGrad(_, unused_grad):
# return [None] # before
return [unused_grad] # I want to modify
TensorFlow uses None to represent 0 for implementation reasons. I don't think it would help you if the returned gradient was true gradient 0, since you want to train. You could use gradient_override_map to substitute gradient of Identity op for Floor as follows
tf.reset_default_graph()
x = tf.Variable(10.)
with tf.get_default_graph().gradient_override_map({"Floor": "Identity"}):
x2 = tf.floor(x)
loss = tf.square(x2)
opt = tf.train.GradientDescentOptimizer(0.1)
train_op = opt.minimize(loss)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for i in range(10):
print(sess.run([loss, train_op]))
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With