Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Keras -- Input Shape for Embedding Layer

I'm trying to implement a convolutional autoencoder in Keras with layers like the one below. My data has 1108 rows and 29430 columns.

def build(features, embedding_dims, maxlen, filters, kernel_size):
    m = keras.models.Sequential()

    m.add(Embedding(features, embedding_dims, input_length=maxlen))
    m.add(Dropout(0.2))

    m.add(Conv1D(filters, kernel_size, padding='valid', activation='relu', strides=1, input_shape=(len(xx), features) ))
    m.add(MaxPooling1D())

    m.add(Conv1D(filters, kernel_size, padding='valid', activation='relu', strides=1, input_shape=(None, len(xx), features) ))
    m.add(UpSampling1D())

    m.summary()
    m.compile(optimizer="adagrad", loss='mse', metrics=['accuracy'])
    return m

early = keras.callbacks.EarlyStopping(
        monitor='val_loss', patience=10, verbose=1, mode='min')

model = build(len(xx[0]), 60, 11900, 70, 3)

model.fit(xx, xx, batch_size=4000, nb_epoch=10000,validation_split=0.1, 
    callbacks=[early])

However, I get an error that states ValueError: Error when checking input: expected embedding_1_input to have shape (None, 11900) but got array with shape (1108, 29430). Why would the first layer expect (None, maxlen) rather than the size of the data?

I'll also include my model summary:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 11900, 60)         714000    
_________________________________________________________________
dropout_1 (Dropout)          (None, 11900, 60)         0         
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 11898, 70)         12670     
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 5949, 70)          0         
_________________________________________________________________
conv1d_2 (Conv1D)            (None, 5947, 70)          14770     
_________________________________________________________________
up_sampling1d_1 (UpSampling1 (None, 11894, 70)         0         
=================================================================
Total params: 741,440
Trainable params: 741,440
Non-trainable params: 0
_________________________________________________________________
like image 776
quil Avatar asked Jul 14 '17 20:07

quil


2 Answers

I fixed this particular error by adding an input_shape field to the Embedding layer as follows:

m.add(Embedding(features, embedding_dims, input_length=maxlen, input_shape=(features, ) ))

features is the number of features (29430).

like image 192
quil Avatar answered Sep 17 '22 18:09

quil


Your input into the Embedding layer must be one dimensional, so you would need to reshape your data into this format (,n). Whatever you passed into input_length would need to match the n size.

like image 38
Jeffrey J Avatar answered Sep 18 '22 18:09

Jeffrey J