Using tf.data.Dataset makes saved model bigger

Question

I recently have an issue with saving the model in a bigger size. I am using tensorflow 1.4

Before, I used

tf.train.string_input_producer() and tf.train.batch()

to load images from a text file. And in the training,

tf.train.start_queue_runners() and tf.train.Coordinator()

were used to provide data to the network. In this case, every time I saved the model using

saver.save(sess, checkpoint_path, global_step=iters)

only gave me a small size file, i.e. a file named model.ckpt-1000.data-00000-of-00001 with 1.6MB.

Now, I use

tf.data.Dataset.from_tensor_slices()

to supply images to an input placeholder and the saved model become 290MB. But I don't know why. I suspect the tensorflow saver saved the dataset into the model as well. If so, how to remove them to make it smaller, and only the weights of the network are saved.

This is not network depended because I tried in two networks and they were all like that.

I have googled but unfortunately didn't see any inspiration related to this issue. (Or this is not an issue, just I don't know how do?)

Thank you very much for any idea and help!

Edit

The method I initialised the dataset is:

1.First generated numpy.array dataset:

self.train_hr, self.train_lr = cifar10.load_dataset(sess)

The initial dataset is numpy.array, for example [8000,32,32,3]. I passed sess into this function is because in the function, I did tf.image.resize_images() and use sess.run() to generate numpy.array. The returns self.train_hr and self.train_lr are numpy.array in shape [8000,64,64,3].

2.Then I created the dataset:

self.img_hr = tf.placeholder(tf.float32)
self.img_lr = tf.placeholder(tf.float32)
dataset = tf.data.Dataset.from_tensor_slices((self.img_hr, self.img_lr))
dataset = dataset.repeat(conf.num_epoch).shuffle(buffer_size=conf.shuffle_size).batch(conf.batch_size)
self.iterator = dataset.make_initializable_iterator()
self.next_batch = self.iterator.get_next()

3.Then I initialised network and dataset, did the training and saved model:

self.labels = tf.placeholder(tf.float32,
                                     shape=[conf.batch_size, conf.hr_size, conf.hr_size, conf.img_channel])
self.inputs = tf.placeholder(tf.float32,
                                     shape=[conf.batch_size, conf.lr_size, conf.lr_size, conf.img_channel])
self.net = Net(self.labels, self.inputs, mask_type=conf.mask_type,
                       is_linear_only=conf.linear_mapping_only, scope='sr_spc')

sess.run(self.iterator.initializer,
                         feed_dict={self.img_hr: self.train_hr, self.img_lr: self.train_lr})
while True:
    hr_img, lr_img = sess.run(self.next_batch)
    _, loss, summary_str = sess.run([train_op, self.net.loss, summary_op],
                                    feed_dict={self.labels: hr_img, self.inputs: lr_img})
    ...
    ...
    checkpoint_path = os.path.join(conf.model_dir, 'model.ckpt')
    saver.save(sess, checkpoint_path, global_step=iters)

All the sess are the same instance.

Answer 1

I suspect you created a tensorflow constant tf.constant out of your dataset, which would explain why the dataset gets stored with the graph. There is an initializeable dataset which let's you feed in the data using feed_dict at runtime. It's a few extra lines of code to configure but it's probably what you wanted to use.

https://www.tensorflow.org/programmers_guide/datasets

Note that constants get created for you automatically in the Python wrapper. The following statements are equivalent:

tf.Variable(42)
tf.Variable(tf.constant(42))