How to use multi threading in keras/tensorflow when fitting a model?

Question

I have a CPU with 20 cores and I am trying to use all the cores to fit a model. I set a tf session with intra_op_parallelism_threads=20 and called model.fit within the same tf session.

The python process utilizes 2000% CPU (as stated by top). However, when comparing the following code with single core configuration (intra_op_parallelism_threads=1) I get the same learning rate.

from keras.layers import Dense, Activation, Dropout
from keras.layers import Input, Conv1D
import numpy as np
from keras.layers.merge import concatenate
from keras.models import Model

import tensorflow as tf
from keras.backend import tensorflow_backend as K

with tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads=20)) as sess:
        K.set_session(sess)

        size=20
        batch_size=16
        def xor_data_generator():
                while True:
                        data1 = np.random.choice([0, 1], size=(batch_size, size,size))
                        data2 = np.random.choice([0, 1], size=(batch_size, size,size))
                        labels  = np.bitwise_xor(data1, data2)
                        yield ([data1, data2], np.array(labels))

        a = Input(shape=(size,size))
        b = Input(shape=(size,size))
        merged = concatenate([a, b])
        hidden = Dense(2*size)(merged)
        conv1 = Conv1D(filters=size*16, kernel_size=1, activation='relu')(hidden)
        hidden = Dropout(0.1)(conv1)
        outputs = Dense(size, activation='sigmoid')(hidden)

        model = Model(inputs=[a, b], outputs=outputs)
        model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
        model.fit_generator(xor_data_generator(), steps_per_epoch=batch_size, epochs=10000) 

Note that I can't use multi_gpu_model, because I have a system with only 20 CPU cores.

How can I distribute model.fit_generator(xor_data_generator(), steps_per_epoch=batch_size, epochs=10000) on different cores simultaneously?

Answer 1

Have a look at Keras' Sequence object to write your custom generator. It is the underlying object of the ImageDataGenerator to yield image data. The docs contain boilerplate code that you can adapt. If you use it, you can set the use_multiprocessing argument of fit.generator() to True. See also this answer.