Tensorboard eval.py IOU for object detection

Question

I used the ssd_mobilenet_v1_coco from detection model zoo in tensorflow object detection. I am currently training the model by running

python legacy/train.py --logtostderr --train_dir=trainingmobile/ --pipeline_config_path=trainingmobile/pipeline.config

I want to run an evaluation job by running eval.py to get other metrics like IOU and PR Curve but I don't know how to do that. I am able to run the command

python legacy/eval.py \
--logtostderr \
--checkpoint_dir= path/to/checkpoint \
--eval_dir= path/to/eval \
--pipeline_config_path= path/to/config

then I ran the command

tensorboard --logdir=path/to/eval

The tensorboard shows only the test image output. How can i get other metrics like IOU and PR Curve?

Answer 1

First of all, I'd highly recommend you to use the newer model_main.py script for training and evaluation combined. You can use it as shown below:

python object_detection/model_main.py \
   --pipeline_config_path=path/to/config \
   --model_dir=path/to/train_dir \
   --num_train_steps=NUM_TRAIN_STEPS \
   --num_eval_steps=NUM_EVAL_STEPS \
   --alsologtostderr

It combines training and evaluation and you can enter tensorboard with

tensorboard -logdir=path/to/train_dir

Tensorboard will not only disply the training process, it will also show your progress over your validation set. They use the COCO metric as default metric!

To your original problem: Maybe you should change the eval settings in your config file to larger numbers:

eval_config: {
  num_examples: 8000
  # Note: The below line limits the evaluation process to 10 evaluations.
  # Remove the below line to evaluate indefinitely.
  max_evals: 10}

If you'll use the model_main.py script, the number of evaluation will be set by the flags.

Good to know: The info output of tnesorflow is disabled in the newer model_main.py script. You can enable it by adding

tf.logging.set_verbosity(tf.logging.INFO)

after the import section.