Tensorflow Estimator: Cache bottlenecks

Question

When following the tensorflow image classification tutorial, at first it caches the bottleneck of each image:

def: cache_bottlenecks())

I have rewritten the training using tensorflow's Estimator. This really simplified all the code. However I want to cache the bottleneck features here.

Here is my model_fn. I want to cache the results of the dense layer so I can make changes to the actual training without having to compute the bottlenecks each time.

How can I accomplish that?

def model_fn(features, labels, mode, params):
    is_training = mode == tf.estimator.ModeKeys.TRAIN

    num_classes = len(params['label_vocab'])

    module = hub.Module(params['module_spec'], trainable=is_training and params['train_module'])
    bottleneck_tensor = module(features['image'])

    with tf.name_scope('final_retrain_ops'):
        logits = tf.layers.dense(bottleneck_tensor, units=num_classes, trainable=is_training)  # save this?

    def train_op_fn(loss):
        optimizer = tf.train.AdamOptimizer()
        return optimizer.minimize(loss, global_step=tf.train.get_global_step())

    head = tf.contrib.estimator.multi_class_head(n_classes=num_classes, label_vocabulary=params['label_vocab'])

    return head.create_estimator_spec(
        features, mode, logits, labels, train_op_fn=train_op_fn
    )

Answer 1

TF cannot work as you code. You should:

1. Export bottleneck to file from the raw net.
2. Use bottleneck result as input, use another net to train your data.