Kernel Crashes while Loading data using tf.data.Dataset.take() from a CSV file

Question

I wanted to Load a CSV file with a Target column and 25 feature columns.I have loaded it via pd.read_csv() as a pandas.Dataframe:

import pandas as pd
import tensorflow as tf


data = pd.read_csv("./data.csv")
data = data.astype('float64')
data.shape #returns (6500, 26)
y_train = data.pop('target')
y_train.shape #returns (6500,)

Then I used the standard Tensorflow 2.0 procedure to read values from pandas.Dataframe:

dataset = tf.data.Dataset.from_tensor_slices((data.values, y_train.values))

As the Docs said,I have to load features and targets seperately from the TensorflowSliceDataset. But as soon as i run the for loop, it freezes for couple of seconds and suddenly kernel dies without any specific reason.

for feat, targ in dataset.take(1):
  print ('Features: {}, Target: {}'.format(feat, targ))

I have tried to run the code without the for loop but the same thing happens with:

tf.constant(data['feature-1'])

I don't know what is causing this problem. I have also re-installed the pandas as well.

Answer 1

I don't quite know what your dataset is but it seems it's a compact one. As written in Load CSV data documentation, You don't need to make a tf.data.Dataset object in order to feed it to the model. You can directly train your model with your data and y_train like this:

import pandas as pd
import tensorflow as tf


data = pd.read_csv("./data.csv")
data = data.astype('float64')
data.shape #returns (6500, 26)
data.head() #Just for extra inspection of your data
y_train = data.pop('target')
y_train.shape #returns (6500,)

my_model= tf.keras.Sequential([
  layers.Dense(64),
  layers.Dense(1)
])

my_model.compile(loss = 'mse',
                      optimizer = 'adam',
                      metrics=['accuracy'])
my_model.fit(data,y_train,epochs=10) #starts training your model

I've used the same model shape from Load CSV data documentation, so you can compare the code later on.

Now , for the tf.data.Dataset.take(count) function, this will return only count elements of the dataset. for example (using abalone dataset mentioned in the official docs):

abalone_dataset = tf.data.Dataset.from_tensor_slices((abalone_features.values, abalone_labels.values))
for feature , label in abalone_dataset.take(1):
  print(f'feature: %s and label: %s' % (feature, label))

#This will reutrn: 
#feature: tf.Tensor([0.435 0.335 0.11  0.334 0.136 0.077 0.097], shape=(7,), dtype=float64) 
#and label: tf.Tensor(7, shape=(), dtype=int64)

So it's basically doing the loop once. Try this code instead:

feature , label  = next(iter(data.take(1)))
print(feature, label)

And also you can check for None types in your pd.Dataframe:

abalone_train.where(lambda x: x~=None).shape
#Should return same shape as your original dataframe

I hope this would fix your problem. Have a nice day.👋