Menu
I have a TensorFlow computational graph for a loss tensor L that depends on 2 tf.Variables, A and B.
I'd like to run gradient ascent on variable A (A+=gradient of L wrt A) while holding B fixed, and vice versa - running gradient ascent on B (B+=gradient of L wrt B) while holding A fixed. How do I do this?
837
If you want to keep variable cells constant, you can lock them before copying the formula. In this example, Column C is the result of the multiplication of values from A2 (2) and Column B. If you copy the formula down Column C, both cell references will change (A2 to A3 and B2 to B3, etc.).
A controlled or constant variable does not change throughout the course of an experiment. It is vitally important that every scientific experiment include a controlled variable; otherwise, the conclusions of an experiment are impossible to understand.
Calling minimize() takes care of both computing the gradients and applying them to the variables. If you want to process the gradients before applying them you can instead use the optimizer in three steps: Compute the gradients with tf. GradientTape .
tf.stop_gradient(tensor) might be what you are looking for. The tensor will be treated as constant for gradient computation purposes. You can create two losses with different parts treated as constants.
The other option (and often better) would be to create 2 optimizers but explicitly optimize only subsets of variables, e.g.
train_a = tf.train.GradientDescentOptimizer(0.1).minimize(loss_a, var_list=[A])
train_b = tf.train.GradientDescentOptimizer(0.1).minimize(loss_b, var_list=[B])
and you can iterate between them on the updates.
59
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
