Tensorflow logging messages do not appear

Question

I use tensorflow 1.2.0 installed with pip install.

When I run samples that include

import logging
tf.logging.set_verbosity(tf.logging.INFO)

the logging messages of the form

logging.info('TEST')

do not appear in the terminal output, even with the flag --tostderr.

According to this answer I also tried

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '0'

but still the problem persists. Any ideas?

Answer 1

TF Logging Basics:

So there is a lot of confusion around tensorflow logging, and it is really not well documented. I landed here a few times in my searches, so it seems to be a good place to post an answer.

After some research and experimentation with Ubuntu and Windows (more than I had planned), this is what I got:

There are two flags, similarly named, but with somewhat different semantics:

• TF_CPP_MIN_LOG_LEVEL - which has 3 or 4 basic levels - low numbers = more messages.
  • 0 outputs Information, Warning, Error, and Fatals (default)
  • 1 outputs Warning, and above
  • 2 outputs Errors and above.
  • etc... I didn't check edge cases
• TF_CPP_MIN_VLOG_LEVEL - which causes very very many extra Information errors - really for debugging only - low numbers = less messages.
  • 3 Outputs lots and lots of stuff
  • 2 Outputs less
  • 1 Outputs even less
  • 0 Outputs nothing extra (default)

Additional Notes:

• Since all the VLOG messages are Informational, then LOG needs to be set at 0 for you to see them. Fortunately that is the default.
• These errors go to the standard error so you can redirect them with something like:
  • python tf-program.py &>mylog.log
• These are supposed to be picked up by the os module so you should be able to set them in the environment
• Without the VLOG and with no GPU there are not that many information messages, so you can think logging is not working when it really is.

Windows:

• Except python's os module did not pick them up under Windows. Python never loved Windows...
  • this code sequence works for me in Windows (and would surely work in Linux):
    • import os
    • os.environ['TF_CPP_MIN_LOG_LEVEL'] = '0'
    • os.environ['TF_CPP_MIN_VLOG_LEVEL'] = '3'
    • import tensorflow as tf

Linux:

• Under Linux (bash) you can specify these conveniently on the command line, so with something like:
  • TF_CPP_MIN_VLOG_LEVEL=3 python tf-program.py

FWIW, I tested on TensorFlow 1.7 with this tutorial:

https://github.com/tensorflow/models/tree/master/tutorials/image/mnist

And this is what it looks like:

Answer 2

There are really two logging systems in tensorflow: one in the C++ core (specifically tensorflow/core/platform/default/logging.{h,cc}) and the other in the Python bindings. The two systems are independent.

The one in the Python binding plays nicely with the standard Python logging module. The C++ logger is indepedently controlled by the TF_CPP_MIN_LOG_LEVEL environment variables mentioned in previous answers.

The following Python (that I locally called logtest.py) demonstrates the two systems' independence.

"""
Demonstrate independence of TensorFlow's two logging subsystems.
"""

import argparse
import tensorflow as tf
import logging

_logger = logging.getLogger( "tensorflow" )

parser = argparse.ArgumentParser( description="Demo TensorFlow logging" )

parser.add_argument("-v","--verbosity",
    default="",
    choices=['DEBUG','INFO','WARNING','ERROR','CRITICAL'],
    help="One of {DEBUG,INFO,WARNING,ERROR,CRITICAL}" )

args = parser.parse_args()

print( "Initial Python effective log level:", _logger.getEffectiveLevel() )

# If user provided an explicit Python level, set it.

if args.verbosity:
    _logger.setLevel( args.verbosity   )
    print( " ...new Python effective log level:", _logger.getEffectiveLevel() ) # ...and confirm the change.

_logger.debug(    "   DEBUG messages are emitted" )
_logger.info(     "    INFO messages are emitted" )
_logger.warn(     " WARNING messages are emitted" )
_logger.error(    "   ERROR messages are emitted" )
_logger.critical( "CRITICAL messages are emitted" )

with tf.Session() as s:
    pass # ...just to trigger TensorFlow into action to generate logging.

Running...

TF_CPP_MIN_LOG_LEVEL=0 python3 logtest.py -v CRITICAL

...shows that Python can't silence the core logging system, and

TF_CPP_MIN_LOG_LEVEL=5 python3 logtest.py -v DEBUG

...shows that the core system can't silence Python.