Menu
How to append new data (e.g. pairs of images and labels) to an already existing tfrecord file?
The class tf.python_io.TFRecordWriter does not seem to have any option for that.
This question may also be reformulated to how to concatenate tfrecord files.
625
TFRecordReader() file = tf. train. string_input_producer("record. tfrecord") _, serialized_record = reader.
The TFRecord format is a simple format for storing a sequence of binary records. Protocol buffers are a cross-platform, cross-language library for efficient serialization of structured data. Protocol messages are defined by . proto files, these are often the easiest way to understand a message type.
Ideally, you should shard the data to ~10N files, as long as ~X/(10N) is 10+ MBs (and ideally 100+ MBs). If it is less than that, you might need to create fewer shards to trade off parallelism benefits and I/O prefetching benefits."
According to the comments in the ticket I opened this won't be implemented, soon.
187
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
