Menu
dataset = dataset.batch(50)
dataset = dataset.prefetch(buffer_size=1)
Is it prefetch 1 batch or 1 element?
Per the API document in tensorflow, the buffer_size is the max num of elements prefetch. But it seems it is num of batch after batching the dataset.
678
buffer_size: A tf. int64 scalar tf. Tensor, representing the maximum number elements that will be buffered when prefetching.
prefetch transformation. It can be used to decouple the time when data is produced from the time when data is consumed. In particular, the transformation uses a background thread and an internal buffer to prefetch elements from the input dataset ahead of the time they are requested.
With that knowledge, from_tensors makes a dataset where each input tensor is like a row of your dataset, and from_tensor_slices makes a dataset where each input tensor is column of your data; so in the latter case all tensors must be the same length, and the elements (rows) of the resulting dataset are tuples with one ...
The tf. data API enables you to build complex input pipelines from simple, reusable pieces. For example, the pipeline for an image model might aggregate data from files in a distributed file system, apply random perturbations to each image, and merge randomly selected images into a batch for training.
Since you are using dataset.prefetch(buffer_size=1) after dataset.batch(), it means that it will prefetch 1 batch.
181
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
