I am trying train an estimator with a generator, but I want to feed this estimator with a package of samples for each iteration. I show the code:
def _generator():
for i in range(100):
feats = np.random.rand(4,2)
labels = np.random.rand(4,1)
yield feats, labels
def input_func_gen():
shapes = ((4,2),(4,1))
dataset = tf.data.Dataset.from_generator(generator=_generator,
output_types=(tf.float32, tf.float32),
output_shapes=shapes)
dataset = dataset.batch(4)
# dataset = dataset.repeat(20)
iterator = dataset.make_one_shot_iterator()
features_tensors, labels = iterator.get_next()
features = {'x': features_tensors}
return features, labels
x_col = tf.feature_column.numeric_column(key='x', shape=(4,2))
es = tf.estimator.LinearRegressor(feature_columns=[x_col],model_dir=tf_data)
es = es.train(input_fn=input_func_gen,steps = None)
When I run this code, it raises this error:
raise ValueError(err.message)
ValueError: Dimensions must be equal, but are 2 and 3 for 'linear/head/labels/assert_equal/Equal' (op: 'Equal') with input shapes: [2], [3].
How do I have to call to this structure??
thx!!!
It is recommended using pre-made Estimators when just getting started. To write a TensorFlow program based on pre-made Estimators, you must perform the following tasks: Create one or more input functions. Define the model's feature columns.
To iterate over the dataset several times, use . repeat() . We can enumerate each batch by using either Python's enumerator or a build-in method.
Prefetching. Prefetching overlaps the preprocessing and model execution of a training step. While the model is executing training step s , the input pipeline is reading the data for step s+1 . Doing so reduces the step time to the maximum (as opposed to the sum) of the training and the time it takes to extract the data ...
estimator —a high-level TensorFlow API. Estimators encapsulate the following actions: Training. Evaluation. Prediction.
The batch size is automatically computed and added to the tensors shapes by Tensorflow, so it doesn't have to be done manually. Your generator should also be defined to output single samples.
Assuming the 4
in position 0 of your shapes are for the batch size, then:
import tensorflow as tf
import numpy
def _generator():
for i in range(100):
feats = numpy.random.rand(2)
labels = numpy.random.rand(1)
yield feats, labels
def input_func_gen():
shapes = ((2),(1))
dataset = tf.data.Dataset.from_generator(generator=_generator,
output_types=(tf.float32, tf.float32),
output_shapes=shapes)
dataset = dataset.batch(4)
# dataset = dataset.repeat(20)
iterator = dataset.make_one_shot_iterator()
features_tensors, labels = iterator.get_next()
features = {'x': features_tensors}
return features, labels
x_col = tf.feature_column.numeric_column(key='x', shape=(2))
es = tf.estimator.LinearRegressor(feature_columns=[x_col])
es = es.train(input_fn=input_func_gen,steps = None)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With