Tensorflow classifier.export_savedmodel (Beginner)

Question

I know about the "Serving a Tensorflow Model" page

https://www.tensorflow.org/serving/serving_basic

but those functions assume you're using tf.Session() which the DNNClassifier tutorial does not... I then looked at the api doc for DNNClassifier and it has an export_savedmodel function (the export function is deprecated) and it seems simple enough but I am getting a "'NoneType' object is not iterable" error... which is suppose to mean I'm passing in an empty variable but I'm unsure what I need to change... I've essentially copied and pasted the code from the get_started/tflearn page on tensorflow.org but then added

  directoryName = "temp"

  def serving_input_fn():
    print("asdf")

  classifier.export_savedmodel(
    directoryName,
    serving_input_fn
  )

just after the classifier.fit function call... the other parameters for export_savedmodel are optional I believe... any ideas?

Tutorial with Code: https://www.tensorflow.org/get_started/tflearn#construct_a_deep_neural_network_classifier

API Doc for export_savedmodel https://www.tensorflow.org/api_docs/python/tf/contrib/learn/DNNClassifier#export_savedmodel

Answer 1

There are two kind of TensorFlow applications:

• The functions that assume you are using tf.Session() are functions from "low level" Tensorflow examples, and
• the DNNClassifier tutorial is a "high level" Tensorflow application.

I'm going to explain how to export "high level" Tensorflow models (using export_savedmodel).

The function export_savedmodel requires the argument serving_input_receiver_fn, that is a function without arguments, which defines the input from the model and the predictor. Therefore, you must create your own serving_input_receiver_fn, where the model input type match with the model input in the training script, and the predictor input type match with the predictor input in the testing script.

On the other hand, if you create a custom model, you must define the export_outputs, defined by the function tf.estimator.export.PredictOutput, which input is a dictionary that define the name that has to match with the name of the predictor output in the testing script.

For example:

TRAINING SCRIPT

def serving_input_receiver_fn():
    serialized_tf_example = tf.placeholder(dtype=tf.string, shape=[None], name='input_tensors')
    receiver_tensors      = {"predictor_inputs": serialized_tf_example}
    feature_spec          = {"words": tf.FixedLenFeature([25],tf.int64)}
    features              = tf.parse_example(serialized_tf_example, feature_spec)
    return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

def estimator_spec_for_softmax_classification(logits, labels, mode):
    predicted_classes = tf.argmax(logits, 1)
    if (mode == tf.estimator.ModeKeys.PREDICT):
        export_outputs = {'predict_output': tf.estimator.export.PredictOutput({"pred_output_classes": predicted_classes, 'probabilities': tf.nn.softmax(logits)})}
        return tf.estimator.EstimatorSpec(mode=mode, predictions={'class': predicted_classes, 'prob': tf.nn.softmax(logits)}, export_outputs=export_outputs) # IMPORTANT!!!
    onehot_labels = tf.one_hot(labels, 31, 1, 0)
    loss          = tf.losses.softmax_cross_entropy(onehot_labels=onehot_labels, logits=logits)
    if (mode == tf.estimator.ModeKeys.TRAIN):
        optimizer = tf.train.AdamOptimizer(learning_rate=0.01)
        train_op  = optimizer.minimize(loss, global_step=tf.train.get_global_step())
        return tf.estimator.EstimatorSpec(mode, loss=loss, train_op=train_op)
    eval_metric_ops = {'accuracy': tf.metrics.accuracy(labels=labels, predictions=predicted_classes)}
    return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)

def model_custom(features, labels, mode):
    bow_column           = tf.feature_column.categorical_column_with_identity("words", num_buckets=1000)
    bow_embedding_column = tf.feature_column.embedding_column(bow_column, dimension=50)   
    bow                  = tf.feature_column.input_layer(features, feature_columns=[bow_embedding_column])
    logits               = tf.layers.dense(bow, 31, activation=None)
    return estimator_spec_for_softmax_classification(logits=logits, labels=labels, mode=mode)

def main():
    # ...
    # preprocess-> features_train_set and labels_train_set
    # ...
    classifier     = tf.estimator.Estimator(model_fn = model_custom)
    train_input_fn = tf.estimator.inputs.numpy_input_fn(x={"words": features_train_set}, y=labels_train_set, batch_size=batch_size_param, num_epochs=None, shuffle=True)
    classifier.train(input_fn=train_input_fn, steps=100)
    full_model_dir = classifier.export_savedmodel(export_dir_base="C:/models/directory_base", serving_input_receiver_fn=serving_input_receiver_fn)

TESTING SCRIPT

def main():
    # ...
    # preprocess-> features_test_set
    # ...
    with tf.Session() as sess:
        tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], full_model_dir)
        predictor   = tf.contrib.predictor.from_saved_model(full_model_dir)
        model_input = tf.train.Example(features=tf.train.Features( feature={"words": tf.train.Feature(int64_list=tf.train.Int64List(value=features_test_set)) })) 
        model_input = model_input.SerializeToString()
        output_dict = predictor({"predictor_inputs":[model_input]})
        y_predicted = output_dict["pred_output_classes"][0]

(Code tested in Python 3.6.3, Tensorflow 1.4.0)

Answer 2

If you try to use predictor with tensorflow > 1.6 you can get this Error :

signature_def_key "serving_default". Available signatures are ['predict']. Original error:
No SignatureDef with key 'serving_default' found in MetaGraphDef.

Here is working example which is tested on 1.7.0 :

SAVING :

First you need to define features length in dict format like this:

feature_spec = {'x': tf.FixedLenFeature([4],tf.float32)}

Then you have to build a function which have placeholder with same shape of features and return using tf.estimator.export.ServingInputReceiver

def serving_input_receiver_fn():
    serialized_tf_example = tf.placeholder(dtype=tf.string,
                                         shape=[None],
                                         name='input_tensors')
    receiver_tensors = {'inputs': serialized_tf_example}

    features = tf.parse_example(serialized_tf_example, feature_spec)
    return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

Then just save with export_savedmodel :

classifier.export_savedmodel(dir_path, serving_input_receiver_fn)

full example code:

import os
from six.moves.urllib.request import urlopen

import numpy as np
import tensorflow as tf


dir_path = os.path.dirname('.')

IRIS_TRAINING = os.path.join(dir_path,  "iris_training.csv")
IRIS_TEST = os.path.join(dir_path,   "iris_test.csv") 

feature_spec = {'x': tf.FixedLenFeature([4],tf.float32)}

def serving_input_receiver_fn():
    serialized_tf_example = tf.placeholder(dtype=tf.string,
                                         shape=[None],
                                         name='input_tensors')
    receiver_tensors = {'inputs': serialized_tf_example}

    features = tf.parse_example(serialized_tf_example, feature_spec)
    return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)




def main():
    training_set = tf.contrib.learn.datasets.base.load_csv_with_header(
        filename=IRIS_TRAINING,
        target_dtype=np.int,
        features_dtype=np.float32)
    test_set = tf.contrib.learn.datasets.base.load_csv_with_header(
        filename=IRIS_TEST,
        target_dtype=np.int,
        features_dtype=np.float32)

    feature_columns = [tf.feature_column.numeric_column("x", shape=[4])]


    classifier = tf.estimator.DNNClassifier(feature_columns=feature_columns,
                                          hidden_units=[10, 20, 10],
                                          n_classes=3,
                                          model_dir=dir_path)
  # Define the training inputs
    train_input_fn = tf.estimator.inputs.numpy_input_fn(
      x={"x": np.array(training_set.data)},
      y=np.array(training_set.target),
      num_epochs=None,
      shuffle=True)

  # Train model.
    classifier.train(input_fn=train_input_fn, steps=200)


    classifier.export_savedmodel(dir_path, serving_input_receiver_fn)


if __name__ == "__main__":
    main()

Restoring

Now let's restore the model :

import tensorflow as tf 
import os

dir_path = os.path.dirname('.') #current directory
exported_path= os.path.join(dir_path,  "1536315752")

def main():
    with tf.Session() as sess:

        tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], exported_path)

        model_input= tf.train.Example(features=tf.train.Features(feature={
                'x': tf.train.Feature(float_list=tf.train.FloatList(value=[6.4, 3.2, 4.5, 1.5]))        
                })) 

        predictor= tf.contrib.predictor.from_saved_model(exported_path)

        input_tensor=tf.get_default_graph().get_tensor_by_name("input_tensors:0")

        model_input=model_input.SerializeToString()

        output_dict= predictor({"inputs":[model_input]})

        print(" prediction is " , output_dict['scores'])


if __name__ == "__main__":
    main()

Here is Ipython notebook demo example with data and explanation :

Answer 3

There are two possible questions and answers possible. First you encounter a missing session for the DNNClassifier which uses the more higher level estimators API (as opposed to the more low level API's where you manipulate the ops yourself). The nice thing about tensorflow is that all high and low level APIs are more-or-less interoperable, so if you want a session and do something with that session, it is as simple as adding:

sess = tf.get_default_session()

The you can start hooking in the remainder of the tutorial.

The second interpretation of your question is, what about the export_savedmodel, well actually export_savedmodel and the sample code from the serving tutorial try to achieve the same goal. When you are training your graph you set up some infrastructure to feed input to the graph (typically batches from a training dataset) however when you switch to 'serving' you will often read your input from somewhere else, and you need some separate infrastructure which replaces the input of the graph used for training. The bottomline is that the serving_input_fn() which you filled with a print should in essence return an input op. This is also said in the documentation:

serving_input_fn: A function that takes no argument and returns an InputFnOps.

Hence instead of print("asdf") it should do something similar as adding an input chain (which should be similar to what builder.add_meta_graph_and_variables is also adding).

Examples of serving_input_fn()'s can for example be found (in the cloudml sample)[https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/census/customestimator/trainer/model.py#L240]. Such as the following which serves input from JSON:

def json_serving_input_fn():
  """Build the serving inputs."""
  inputs = {}
  for feat in INPUT_COLUMNS:
    inputs[feat.name] = tf.placeholder(shape=[None], dtype=feat.dtype)
  return tf.estimator.export.ServingInputReceiver(inputs, inputs)