I have a .tfrecords
file of the Ubuntu Dialog Corpus. I am trying to read in the whole dataset so that I can split the contexts and utterances into batches. Using tf.parse_single_example
I was able to read in a single example. I tried using tf.parse_example
but I get the following error
ValueError: Shape must be rank 1 but is rank 0 for 'ParseExample/ParseExample' (op: 'ParseExample') with input shapes: [], [0], [], [], [], [], [], [0], [0], [0], [0], [0].
I am not sure what to make of it. The code I used to get to the error -
import tensorflow as tf
TRAIN_FILE_TFREC = 'data/train.tfrecords'
filename_queue = tf.train.string_input_producer([TRAIN_FILE_TFREC])
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_example(serialized_example,
features = {
"context" : tf.FixedLenFeature([160], tf.int64),
"context_len" : tf.FixedLenFeature([1], tf.int64),
"utterance" : tf.FixedLenFeature([80], tf.int64),
"utterance_len" : tf.FixedLenFeature([1], tf.int64),
"label" : tf.FixedLenFeature([1], tf.int64)
})
Any ideas
to use tf.parse_example you need to first batch the examples:
batch = tf.train.batch([serialized_example], num_examples, capacity=num_examples)
parsed_examples = tf.parse_example(batch, feature_spec)
I had a similar problem just now. Try putting brackets around the serialized_example to turn it into a list:
features = tf.parse_example([serialized_example],
features = {
"context" : tf.FixedLenFeature([160], tf.int64),
"context_len" : tf.FixedLenFeature([1], tf.int64),
"utterance" : tf.FixedLenFeature([80], tf.int64),
"utterance_len" : tf.FixedLenFeature([1], tf.int64),
"label" : tf.FixedLenFeature([1], tf.int64)
})
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With