Does the TensorFlow backend of Keras rely on the eager execution?

Question

Does the TensorFlow backend of Keras rely on the eager execution?

If it isn't the case, can I build a TensorFlow graph based on Keras and TensorFlow operations, then train the whole model using Keras high-level API?

Answer 1

It is for a research purpose which I can't present here.

That makes it really difficult to answer your question. It would be better if you could find a toy example -- unrelated with your research -- of what you want and we try to build something from there.

Does the TensorFlow backend of Keras rely on the eager execution?

No, it doesn't. Keras was built before eager execution introduction. Keras (the one inside tf) can, however, work in eager execution mode (see fchollet's answer).

can I build a TensorFlow graph and combine it with a Keras model then train them jointly using Keras high-level API?

I'm not sure what you mean by "build a TensorFlow graph", because a graph already exists whenever you use keras. If you are talking about adding a bunch of operations to the existing graph, then it's definitely possible. You just need to wrap it up with a Lambda layer, just like you'd do if using Keras on symbolic mode:

import tensorflow as tf
from sacred import Experiment

ex = Experiment('test-18')

tf.enable_eager_execution()


@ex.config
def my_config():
    pass


@ex.automain
def main():
    (x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

    x_train, x_test = (e.reshape(e.shape[0], -1) for e in (x_train, x_test))
    y_train, y_test = (tf.keras.utils.to_categorical(e) for e in (y_train, y_test))

    def complex_tf_fn(x):
        u, v = tf.nn.moments(x, axes=[1], keep_dims=True)
        return (x - u) / tf.sqrt(v)

    with tf.device('/cpu:0'):
        model = tf.keras.Sequential([
            tf.keras.layers.Lambda(complex_tf_fn, input_shape=[784]),
            tf.keras.layers.Dense(1024, activation='relu'),
            tf.keras.layers.Lambda(complex_tf_fn),
            tf.keras.layers.Dense(10, activation='softmax')
        ])
        model.compile(optimizer=tf.train.AdamOptimizer(),
                      loss='categorical_crossentropy')

        model.fit(x_train, y_train,
                  epochs=10,
                  validation_data=(x_test, y_test),
                  batch_size=1024,
                  verbose=2)

python test-18.py with seed=21

INFO - test-18 - Running command 'main'
INFO - test-18 - Started
Train on 60000 samples, validate on 10000 samples
Epoch 1/10
 - 9s - loss: 3.4012 - val_loss: 1.3575
Epoch 2/10
 - 9s - loss: 0.9870 - val_loss: 0.7270
Epoch 3/10
 - 9s - loss: 0.6097 - val_loss: 0.6071
Epoch 4/10
 - 9s - loss: 0.4459 - val_loss: 0.4824
Epoch 5/10
 - 9s - loss: 0.3352 - val_loss: 0.4436
Epoch 6/10
 - 9s - loss: 0.2661 - val_loss: 0.3997
Epoch 7/10
 - 9s - loss: 0.2205 - val_loss: 0.4048
Epoch 8/10
 - 9s - loss: 0.1877 - val_loss: 0.3788
Epoch 9/10
 - 9s - loss: 0.1511 - val_loss: 0.3506
Epoch 10/10
 - 9s - loss: 0.1304 - val_loss: 0.3330
INFO - test-18 - Completed after 0:01:31

Process finished with exit code 0