How to detect which variable is 'nonetype' in tensorflow

Question

I am using TensorFlow to create a new model, which includes a dynamic loop. I am using tf.while_loop to implement this instance. One issue I encounter is that:

AttributeError: 'NoneType' object has no attribute 'back_prop'

This problem appears while doing

gradients = tf.gradients(self.loss, params)

Then, I try to print all the params and it turns out every parameter has a shape. I think if there is a nonetype parameter, its shape should be None as well? On the other hand, is there any other method that could help me detect which variable is not assigned or like []?

Here is full trackback:

  Traceback (most recent call last):
  File "main.py", line 125, in <module>
    tf.app.run()
  File "/usr/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
    sys.exit(main(sys.argv))
  File "main.py", line 119, in main
    train()# if FLAGS.train:
  File "main.py", line 95, in train
    model = create_model(sess, False)
  File "main.py", line 75, in create_model
    forward_only=False)
  File "/home/sniu/lab/ai_lab/DMN-tensorflow/models/DMN.py", line 248, in __init__
    gradients = tf.gradients(self.loss, params)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/gradients.py", line 481, in gradients
    in_grads = _AsList(grad_fn(op, *out_grads))
  File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_grad.py", line 181, in _EnterGrad
    if not grad_ctxt.back_prop:
AttributeError: 'NoneType' object has no attribute 'back_prop'

Answer 1

NoneType just means that the value is None

>>> item = None
>>> item.value
Traceback (most recent call last):
  File "<stdin>", line 1 in <module>
AttributeError: 'NoneType' object has no attribute 'value'

You can see if you call type on None

>>> type(None)
<type 'NoneType'>

None is kind of a special value in python. It's a singleton object. It's an instance of NoneType, and all None's are the exact same object.

Generally, to prevent these types of errors people either test if the value is None first, or they wrap the expression in a try/except block

if item is not None:
    print item.back_prop

Or using a try/except

try:
    item.back_prop
except AttributeError:
    pass

Be aware that the try/except block may suppress other AttributeErrors unrelated to item being None, like if item is some other value that also doesn't have a back_prop attribute. You may want to treat that situation differently than if item is None.

Answer 2

Based on this comment, I resolved a similar issue with gradients that were None.

opt = tf.train.RMSPropOptimizer(1e-3)
grads, vars = zip(
    *opt.compute_gradients(loss, var_list=my_varlist))
grads = [g if g is not None else tf.zeros_like(v)
         for g, v in zip(grads, vars)]
optim = opt.apply_gradients(zip(grads, vars))