Keras classification model

Question

I need help to build keras model for classification. I have

Input: 167 points of optical spectrum

Output 11 classes of investigated substance.

But in one data set can be spectre of substance with several substance (for example contains classes 2,3,4). I tried to use categorical_crossentropy, but it is suitable only for non-intersecting classes.

KerasDoc:

Note: when using the categorical_crossentropy loss, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros expect for a 1 at the index corresponding to the class of the sample). In order to convert integer targets into categorical targets, you can use the Keras utility to_categorical:

My code:

model = Sequential()
model.add(Dense(64, input_dim=167))
model.add(Dense(32))
model.add(Dense(11))
model.add(Activation('sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

I tried many models but can not get a good result.

Answer 1

You should probably go well with sigmoid and binary_crossentropy (See here)

PS: This is not your case, but for a categorial_crossentropy, you should ideally use a softmax activation. The softmax outputs things optimized to maximize one class only.

(If anyone would like to complement this answer with a good or better "optimizer", feel free).