Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Gradient of tf.floor is None

tf.floor return None gradient
it means the weights before floor operation won't be update, right?
but I still need the gradient to update weights
Isn't it weird? sometimes we use floor, ceil... but they can't deliver gradient or this is right action
as well as tf.cast, the gradient return none

Here is an issue #897 to discuss the gradient, but I don't understand why return None is good

numa = tf.constant(5.3)  
res1 = numa * 2   
res2 = tf.floor(numa *2)   
print tf.gradients(res1, numa)
# <type 'list'>: [<tf.Tensor 'gradients/mul_grad/Reshape:0' shape=() dtype=float32>]

print tf.gradients(res2, numa)
# None

Can I modify math_grad.py directly? need other action?
something like delete math_grad.pyc?

@ops.RegisterGradient("Floor")
def _FloorGrad(_, unused_grad):
  # return [None]  # before
  return [unused_grad] # I want to modify
like image 506
xxi Avatar asked Sep 10 '26 07:09

xxi


1 Answers

TensorFlow uses None to represent 0 for implementation reasons. I don't think it would help you if the returned gradient was true gradient 0, since you want to train. You could use gradient_override_map to substitute gradient of Identity op for Floor as follows

tf.reset_default_graph()
x = tf.Variable(10.)
with tf.get_default_graph().gradient_override_map({"Floor": "Identity"}):
    x2 = tf.floor(x)
loss = tf.square(x2)
opt = tf.train.GradientDescentOptimizer(0.1)
train_op = opt.minimize(loss)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for i in range(10):
    print(sess.run([loss, train_op]))
like image 147
Yaroslav Bulatov Avatar answered Sep 12 '26 21:09

Yaroslav Bulatov



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!