How estimator of tensorflow load specific steps of model instead of latest one?

Question

We can save many checkpoints of model using Estimator and RunConfig.

classifier eval will use the latest step 200 by default, can I load ckpt-1?

my_checkpointing_config = tf.estimator.RunConfig(
    save_checkpoints_secs = 20*60,  # Save checkpoints every 20 minutes.
    keep_checkpoint_max = 10,       # Retain the 10 most recent checkpoints.
)
classifier = tf.estimator.DNNClassifier(
    feature_columns=my_feature_columns,
    hidden_units=[10, 10],
    n_classes=3,
    model_dir='models/iris',
    config=my_checkpointing_config)



$ ls -1 models/iris
checkpoint
events.out.tfevents.timestamp.hostname
graph.pbtxt
model.ckpt-1.data-00000-of-00001
model.ckpt-1.index
model.ckpt-1.meta
model.ckpt-200.data-00000-of-00001
model.ckpt-200.index
model.ckpt-200.meta

Answer 1

Both tf.estimator.Estimator.evaluate and tf.estimator.Estimator.predict have a checkpoint_path argument. You should be able to supply the path to model.ckpt-1 here to use this checkpoint for evaluation.

Note that this argument was added in a fairly recent TF update (might be 1.7 or 1.8, not quire sure) so if you are using an outdated version you might not have this argument available. There is a hacky alternative: In the model_dir there should be a file called checkpoint. The first line of this file should be

model_checkpoint_path: "model.ckpt-xxxxxx"

where xxxxxx is the number of steps for the latest checkpoint (200 in your case). You can manually change this line to whatever checkpoint you want the Estimator to load. However you will probably want to change it back afterwards or you might run into issues if you ever want to continue training the model.