Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tensorflow - ValueError: Shape must be rank 1 but is rank 0 for 'ParseExample/ParseExample'

I have a .tfrecords file of the Ubuntu Dialog Corpus. I am trying to read in the whole dataset so that I can split the contexts and utterances into batches. Using tf.parse_single_example I was able to read in a single example. I tried using tf.parse_example but I get the following error

ValueError: Shape must be rank 1 but is rank 0 for 'ParseExample/ParseExample' (op: 'ParseExample') with input shapes: [], [0], [], [], [], [], [], [0], [0], [0], [0], [0].

I am not sure what to make of it. The code I used to get to the error -

import tensorflow as tf    
TRAIN_FILE_TFREC = 'data/train.tfrecords'

filename_queue = tf.train.string_input_producer([TRAIN_FILE_TFREC])

reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)

features = tf.parse_example(serialized_example, 
features = {
"context" : tf.FixedLenFeature([160], tf.int64),
"context_len" : tf.FixedLenFeature([1], tf.int64),
"utterance" : tf.FixedLenFeature([80], tf.int64),
"utterance_len" : tf.FixedLenFeature([1], tf.int64),
"label" : tf.FixedLenFeature([1], tf.int64)
})

Any ideas

like image 537
Clock Slave Avatar asked Jan 31 '17 06:01

Clock Slave


2 Answers

to use tf.parse_example you need to first batch the examples:

batch = tf.train.batch([serialized_example], num_examples, capacity=num_examples)
parsed_examples = tf.parse_example(batch, feature_spec)
like image 55
Jon B Avatar answered Nov 10 '22 20:11

Jon B


I had a similar problem just now. Try putting brackets around the serialized_example to turn it into a list:

features = tf.parse_example([serialized_example], 
features = {
"context" : tf.FixedLenFeature([160], tf.int64),
"context_len" : tf.FixedLenFeature([1], tf.int64),
"utterance" : tf.FixedLenFeature([80], tf.int64),
"utterance_len" : tf.FixedLenFeature([1], tf.int64),
"label" : tf.FixedLenFeature([1], tf.int64)
})
like image 3
user2762495 Avatar answered Nov 10 '22 20:11

user2762495