Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multiple outputs in Keras

I have a problem which deals with predicting two outputs when given a vector of predictors. Assume that a predictor vector looks like x1, y1, att1, att2, ..., attn, which says x1, y1 are coordinates and att's are the other attributes attached to the occurrence of x1, y1 coordinates. Based on this predictor set I want to predict x2, y2. This is a time series problem, which I am trying to solve using multiple regresssion. My question is how do I setup keras, which can give me 2 outputs in the final layer.

like image 437
Neelabh Pant Avatar asked May 18 '17 00:05

Neelabh Pant


People also ask

Can neural networks have multiple outputs?

Neural network models can be configured for multi-output regression tasks.

What is multi-output classification?

Multi-output classification is a type of machine learning that predicts multiple outputs simultaneously. In multi-output classification, the model will give two or more outputs after making any prediction. In other types of classifications, the model usually predicts only a single output.

Can linear regression have multiple outputs?

Many machine learning algorithms are designed for predicting a single numeric value, referred to simply as regression. Some algorithms do support multioutput regression inherently, such as linear regression and decision trees.


2 Answers

from keras.models import Model from keras.layers import *      #inp is a "tensor", that can be passed when calling other layers to produce an output  inp = Input((10,)) #supposing you have ten numeric values as input    #here, SomeLayer() is defining a layer,  #and calling it with (inp) produces the output tensor x x = SomeLayer(blablabla)(inp)  x = SomeOtherLayer(blablabla)(x) #here, I just replace x, because this intermediate output is not interesting to keep   #here, I want to keep the two different outputs for defining the model #notice that both left and right are called with the same input x, creating a fork out1 = LeftSideLastLayer(balbalba)(x)     out2 = RightSideLastLayer(banblabala)(x)   #here, you define which path you will follow in the graph you've drawn with layers #notice the two outputs passed in a list, telling the model I want it to have two outputs. model = Model(inp, [out1,out2]) model.compile(optimizer = ...., loss = ....) #loss can be one for both sides or a list with different loss functions for out1 and out2      model.fit(inputData,[outputYLeft, outputYRight], epochs=..., batch_size=...) 
like image 134
Daniel Möller Avatar answered Sep 19 '22 12:09

Daniel Möller


You can make a model with multiple output with

  1. the Functional API

  2. by subclassing tf.keras.Model.

Here's an example of dual outputs (regression and classification) on the Iris Dataset, using the Functional API:

from sklearn.datasets import load_iris from tensorflow.keras.layers import Dense from tensorflow.keras import Input, Model import tensorflow as tf  data, target = load_iris(return_X_y=True) X = data[:, (0, 1, 2)] Y = data[:, 3] Z = target  inputs = Input(shape=(3,), name='input') x = Dense(16, activation='relu', name='16')(inputs) x = Dense(32, activation='relu', name='32')(x) output1 = Dense(1, name='cont_out')(x) output2 = Dense(3, activation='softmax', name='cat_out')(x)  model = Model(inputs=inputs, outputs=[output1, output2])  model.compile(loss={'cont_out': 'mean_absolute_error',                      'cat_out': 'sparse_categorical_crossentropy'},               optimizer='adam',               metrics={'cat_out': tf.metrics.SparseCategoricalAccuracy(name='acc')})  history = model.fit(X, {'cont_out': Y, 'cat_out': Z}, epochs=10, batch_size=8) 

Here's a simplified version:

from sklearn.datasets import load_iris from tensorflow.keras.layers import Dense from tensorflow.keras import Input, Model  data, target = load_iris(return_X_y=True) X = data[:, (0, 1, 2)] Y = data[:, 3] Z = target  inputs = Input(shape=(3,)) x = Dense(16, activation='relu')(inputs) x = Dense(32, activation='relu')(x) output1 = Dense(1)(x) output2 = Dense(3, activation='softmax')(x)  model = Model(inputs=inputs, outputs=[output1, output2])  model.compile(loss=['mae', 'sparse_categorical_crossentropy'], optimizer='adam')  history = model.fit(X, [Y, Z], epochs=10, batch_size=8) 

Here's the same example, subclassing tf.keras.Model and with a custom training loop:

import tensorflow as tf from tensorflow.keras.layers import Dense from tensorflow.keras import Model from sklearn.datasets import load_iris tf.keras.backend.set_floatx('float64') iris, target = load_iris(return_X_y=True)  X = iris[:, :3] y = iris[:, 3] z = target  ds = tf.data.Dataset.from_tensor_slices((X, y, z)).shuffle(150).batch(8)  class MyModel(Model):     def __init__(self):         super(MyModel, self).__init__()         self.d0 = Dense(16, activation='relu')         self.d1 = Dense(32, activation='relu')         self.d2 = Dense(1)         self.d3 = Dense(3, activation='softmax')      def call(self, x, training=None, **kwargs):         x = self.d0(x)         x = self.d1(x)         a = self.d2(x)         b = self.d3(x)         return a, b  model = MyModel()  loss_obj_reg = tf.keras.losses.MeanAbsoluteError() loss_obj_cat = tf.keras.losses.SparseCategoricalCrossentropy()  optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)  loss_reg = tf.keras.metrics.Mean(name='regression loss') loss_cat = tf.keras.metrics.Mean(name='categorical loss')  error_reg = tf.keras.metrics.MeanAbsoluteError() error_cat = tf.keras.metrics.SparseCategoricalAccuracy()  @tf.function def train_step(inputs, y_reg, y_cat):     with tf.GradientTape() as tape:         pred_reg, pred_cat = model(inputs)         reg_loss = loss_obj_reg(y_reg, pred_reg)         cat_loss = loss_obj_cat(y_cat, pred_cat)      gradients = tape.gradient([reg_loss, cat_loss], model.trainable_variables)     optimizer.apply_gradients(zip(gradients, model.trainable_variables))     loss_reg(reg_loss)     loss_cat(cat_loss)      error_reg(y_reg, pred_reg)     error_cat(y_cat, pred_cat)   for epoch in range(50):     for xx, yy, zz in ds:         train_step(xx, yy, zz)      template = 'Epoch {:>2}, SCCE: {:>5.2f},' \                ' MAE: {:>4.2f}, SAcc: {:>5.1%}'     print(template.format(epoch+1,                         loss_cat.result(),                         error_reg.result(),                         error_cat.result()))      loss_reg.reset_states()     loss_cat.reset_states()      error_reg.reset_states()     error_cat.reset_states() 
like image 20
Nicolas Gervais Avatar answered Sep 21 '22 12:09

Nicolas Gervais