How can I use tf.data Datasets in eager execution mode?

Question

In the tf.data talk at the TensorFlow Dev Summit 2018, Derek Murray presented a way to combine the tf.data API with TensorFlow's eager execution mode (at 10:54). I tried out a simplified version of the code shown there:

import tensorflow as tf
tf.enable_eager_execution()

dataset = tf.data.Dataset.from_tensor_slices(tf.random_uniform([50, 10]))
dataset = dataset.batch(5)
for batch in dataset:
    print(batch)

causing

TypeError: 'BatchDataset' object is not iterable

I also tried using dataset.make_one_shot_iterator() and dataset.make_initializable_iterator() to iterate over the dataset, but they result in

RuntimeError: dataset.make_one_shot_iterator is not supported when eager execution is enabled.

and

RuntimeError: dataset.make_initializable_iterator is not supported when eager execution is enabled.

TensorFlow version: 1.7.0, Python version: 3.6

How can you use the tf.data API with eager execution?

Answer 1

With TF 2.1,

You can create an iterator like so:

iterator = iter(dataset)

And get the next batch of values:

batch = iterator.get_next()

Answer 2

make_one_shot_iterator() should work in TensorFlow 1.8, but for now (i.e., for TensorFlow 1.7), do the following:

import tensorflow.contrib.eager as tfe

dataset = tf.data.Dataset.from_tensor_slices(tf.random_uniform([50, 10]))
dataset = dataset.batch(5)
for batch in tfe.Iterator(dataset):
     print(batch)