Add hand-crafted features to Keras sequential model

Question

I have 1D sequences which I want to use as input to a Keras VGG classification model, split in x_train and x_test. For each sequence, I also have custom features stored in feats_train and feats_test which I do not want to input to the convolutional layers, but to the first fully connected layer.

A complete sample of train or test would thus consist of a 1D sequence plus n floating point features.

What is the best way to feed the custom features first to the fully connected layer? I thought about concatenating the input sequence and the custom features, but I do not know how to make them separate inside the model. Are there any other options?

The code without the custom features:

x_train, x_test, y_train, y_test, feats_train, feats_test = load_balanced_datasets()

model = Sequential()
model.add(Conv1D(10, 5, activation='relu', input_shape=(timesteps, 1)))
model.add(Conv1D(10, 5, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.5, seed=789))

model.add(Conv1D(5, 6, activation='relu'))
model.add(Conv1D(5, 6, activation='relu'))
model.add(MaxPooling1D(pool_size=2))
model.add(Dropout(0.5, seed=789))

model.add(Flatten())

model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5, seed=789))
model.add(Dense(2, activation='softmax'))

model.compile(loss='logcosh', optimizer='adam', metrics=['accuracy'])

model.fit(x_train, y_train, batch_size=batch_size, epochs=20, shuffle=False, verbose=1)

y_pred = model.predict(x_test)

Answer 1

Sequential model is not very flexible. You should look into the functional API.

I would try something like this:

from keras.layers import (Conv1D, MaxPool1D, Dropout, Flatten, Dense,
                          Input, concatenate)
from keras.models import Model, Sequential

timesteps = 50
n = 5

def network():
    sequence = Input(shape=(timesteps, 1), name='Sequence')
    features = Input(shape=(n,), name='Features')

    conv = Sequential()
    conv.add(Conv1D(10, 5, activation='relu', input_shape=(timesteps, 1)))
    conv.add(Conv1D(10, 5, activation='relu'))
    conv.add(MaxPool1D(2))
    conv.add(Dropout(0.5, seed=789))

    conv.add(Conv1D(5, 6, activation='relu'))
    conv.add(Conv1D(5, 6, activation='relu'))
    conv.add(MaxPool1D(2))
    conv.add(Dropout(0.5, seed=789))
    conv.add(Flatten())
    part1 = conv(sequence)

    merged = concatenate([part1, features])

    final = Dense(512, activation='relu')(merged)
    final = Dropout(0.5, seed=789)(final)
    final = Dense(2, activation='softmax')(final)

    model = Model(inputs=[sequence, features], outputs=[final])

    model.compile(loss='logcosh', optimizer='adam', metrics=['accuracy'])

    return model

m = network()