Why do we say functional API in Keras is used for non-sequential models?

Question

AFAIK, we still need to create a model and add layers to the model using the functional API. Why people say that the functional AI is used to create non-sequential neural network?

Answer 1

Why people said it is used to non-sequential neural network?

The thing is that with the Sequential Model you are defining a model step by step (sequentially), when you call the .add() method. Whereas, on the Functional API (specifically the Model class) you have more freedom, as you can define different layers that receive different inputs and then instantiate a model with the Model creator using any of those layers (not necessarily in a step by step, or sequential way).

In other words, when calling model = Sequential() you are in that moment instantiating your model object (for which you then add layers and constraints). In the Functional API, you create layers and then instantiate your model by calling model = Model(inputs=in, outputs=out) with your desired input and output layer(s). As you see, both approaches can be equivalent, for example, these two are the same:

from keras.models import Sequential, Model
from keras.layers import Input, Dense, Activation
#---Using the Sequential model
model = Sequential() #Object is instantiated here
model.add(Dense(32, input_dim=784))
model.add(Activation('relu'))
#---Or using the Functional API
a = Input(shape=(784,))
b = Dense(32, activation='relu')(a)
model = Model(inputs=a, outputs=b) #Object is instantiated here

Considering only this, then choosing which way to go depends more on your personal style and coding preferences. Now, there is a major advantage of using the Functional API over the Sequential model, which is that you can share or reuse layers across different models.

When compiling and fitting a model, all its associated layers will be compiled and trained. Thus, any other model that shares such layers will also reflect those changes. This gives you the freedom to do many things, as obtaining sub-models of your network, redefine them, obtain its relative outputs, merge them into more complex models, etc., without having to train again for each of those sub-models.

To make it clearer, here is an example (based on this Keras Autoencoder Blog post) that illustrates what was discussed in the last paragraph:

from keras.layers import Input, Dense
from keras.models import Model
#Create an autoencoder, along with its encoder and decoder model
input_img = Input(shape=(784,))
encoded = Dense(32, activation='relu')(input_img)
decoded = Dense(784, activation='sigmoid')(encoded)

#Here we define our autoencoder model: image -> encoding -> decoded image
autoencoder = Model(input_img, decoded)

#Now here is the advantage of the Funcional API
#We can reuse those layers to obtain an encoder model (image -> encoding)
#as well as a decoder model (encoding -> image)
#but compile all three by just compiling and fitting the Autoencoder model
encoder = Model(input_img, encoded) #see how the 'encoded' layer is output
# create a placeholder for an encoded (32-dimensional) input
encoded_input = Input(shape=(32,))
# retrieve the last layer of the autoencoder model
decoder_layer = autoencoder.layers[-1]
# create the decoder model
decoder = Model(encoded_input, decoder_layer(encoded_input))
#compile and fit with your data
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
autoencoder.fit(X,Y,...)

After this, you will be able to make predictions on the encoder and decoder model individually (to visualize your encoding, for example), as well as make predictions with the autoencoder model as a whole. At this point, doing the following things are equivalent:

#obtain reconstructed representation directly
autoencoder_imgs = autoencoder.predict(x_test)
#obtain reconstructed representation by joining encoder and decoder models
encoder_imgs = encoder.predict(x_test)
decoded_imgs = decoder.predict(encoded_imgs)

Hope this helps. Personally, I always try to use the Functional API instead, regardless if I want or not to reuse or recycle layers, as I find it more verbose, but that is up to you to decide.

Answer 2

Well, "sequential" is not really the most correct term, but it's the name chosen by Keras developers. All models work in sequence, of course.

The difference is:

• The Sequential model is a straight line. You keep adding layers, every new layer takes the output of the previous layer. You cannot make creative graphs with branches.
• The functoinal API Model is completely free to have as many ramifications, inputs and outputs as you need.

Example with the Sequential model:

from keras.models import Sequential 
from keras.layers import *

#you create a model
model = Sequential()

#and you add layers
model.add(SomeKerasLayer(...))
model.add(AnotherKerasLayer(...))

#as you can see, this model is a straight line, you only add layers "sequentially"

Example with the functional API Model:

Now here we start creating really fancy models.

from keras.models import Model
from keras.layers import *

We start by defining input tensors. And we can have any number of inputs! (The sequential model is limited to one input, that you define in the first layer with input_shape).

input1 = Input(inputShape1)

#We can have more inputs if we want!
input2 = Input(inputShape2)
input3 = Input(inputShape3)

We work by creating layers and "calling layers with input tensors".
When we call a layer with an input tensor, we get an output tensor.
And we can create whatever path we want.

#Example: two separate layers taking two separate inputs:
output1 = SomeLayer(...)(input1)
output2 = AnotherLayer(...)(input2)

We can join two branches with different options, such as add, multiply, concatenate, etc.:

#joining the previous tensors output1 and output2
joined1_2 = Concatenate()([output1,output2])

We can reuse the same layers with different inputs, getting different outputs:

aLayer = AKerasLayer(...) #notice I'm creating this layer but not calling it yet

#calling the same layer with two different inputs
output1 = aLayer(joined1_2)
output2 = aLayer(input3)

And finally, we can define the model with as many inputs and outputs we want:

model = Model([input1,input2,input3],[output1, output2])

---

Reusing and associating models

Both models, Sequential and functional API, can be used as if they were layers.

You can call a model with and input tensor and get an output tensor, just like you would do when creating a functional API model:

input1 = Input(shape)
output1 = anExistingSequentialModel(input1)
output2 = anExistingFunctionalModel(input1)

newModel = Model(input1,[output1,output2])

And you can add models in sequential models as well (beware of the branches, it's best to have one input and one output for the added model, since this is a Sequential one)

seqModel = Sequential()
seqModel.add(anotherModel)