Received a label value of 1 which is outside the valid range of [0, 1) - Python, Keras

Question

I am working on a simple cnn classifier using keras with tensorflow background.

def cnnKeras(training_data, training_labels, test_data, test_labels, n_dim):
  print("Initiating CNN")
  seed = 8
  numpy.random.seed(seed)
  model = Sequential()
  model.add(Convolution2D(64, 1, 1, init='glorot_uniform', 
   border_mode='valid',input_shape=(16, 1, 1), activation='relu'))
  model.add(MaxPooling2D(pool_size=(1, 1)))
  model.add(Convolution2D(32, 1, 1, init='glorot_uniform', 
   activation='relu'))
  model.add(MaxPooling2D(pool_size=(1, 1)))
  model.add(Dropout(0.25))
  model.add(Flatten())
  model.add(Dense(128, activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(64, activation='relu'))
  model.add(Dense(1, activation='softmax'))
  # Compile model
  model.compile(loss='sparse_categorical_crossentropy',
              optimizer='adam', metrics=['accuracy'])
  model.fit(training_data, training_labels, validation_data=(
    test_data, test_labels), nb_epoch=30, batch_size=8, verbose=2)

  scores = model.evaluate(test_data, test_labels, verbose=1)
  print("Baseline Error: %.2f%%" % (100 - scores[1] * 100))
  # model.save('trained_CNN.h5')
  return None

It is a binary classification problem, but I keep getting the message Received a label value of 1 which is outside the valid range of [0, 1) which does not make any sense to me. Any suggesstions?

Answer 1

Range [0, 1) means every number between 0 and 1, excluding 1. So 1 is not a value in the range [0, 1).

I am not 100% sure, but the issue could be due to your choice of loss function. For a binary classification, binary_crossentropy should be a better choice.

Answer 2

In the last Dense layer you used model.add(Dense(1, activation='softmax')). Here 1 restricts its value from [0, 1) change its shape to the maximum output label. For eg your output is from label [0,7) then use model.add(Dense(7, activation='softmax'))

Answer 3

Peculiarities of sparse categorical crossentropy

The loss function sparse_categorical_crossentropy interprets the final layer in the context of classifiers as a set of probabilities for each possible class, and the output value as the number of the class. (The Tensorflow/Keras documentation goes into a bit more detail.) So x neurons in output layer are compared against output values in the range from 0 to x-1; and having just one neuron in the output layer is an 'unary' classifier that doesn't make sense.

If it's a classification task where you want to have output data in the form from 0 to x-1, then you can keep sparse categorical crossentropy, but you need to set the number of neurons in the output layer to the number of classes you have. Alternatively, you might encode the output in a one-hot vector and use categorical crossentropy loss function instead of sparse categorical crossentropy.

If it's not a classification task and you want to predict arbitrary real-valued numbers as in a regression, then categorical crossentropy is not a suitable loss function at all.

Answer 4

Cray and Shaili's answer was correct! I had a range of outcomes from 1 to 6, and the line:

tf.keras.layers.Dense(6, activation = 'softmax') 

Produced that error message, saying that things were outside of the range [0,6). I had thought that it was a labels problem (were all values present in both the training and validation label sets?), and was flogging them.

)