Explaining a multi-label target Tensorflow model with SHAP

Question

I have a tensorflow model with 2 target vars that i would like to view its SHAP values as following:

import pandas as pd
import tensorflow as tf
import shap

x_df = pd.DataFrame({'var1':[4, 6, 19, 8],
                         'var2':[7, 21, 5, 12],
                         'var3':[11, 15, 19, 5],
                         'var4':[8, 1, 16, 18]})

target_var = pd.DataFrame({'y1': [12, 4, 6, 8],
                           'y2': [11, 13, 9, 12]})


model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(4, activation='sigmoid'),
    tf.keras.layers.Dense(2)])
model.compile(optimizer='adam', loss="mse")
model.fit(x_df.values, target_var, epochs=1, batch_size=2)

explainer = shap.DeepExplainer(model, x_df)
shap_values = explainer.shap_values(x_df)
shap.summary_plot(shap_values, x_df)

Im using several tutorials that explain that i can insert the tf model straight into the explainer and use it, however the .shap values returns an error as following:

AttributeError: 'tuple' object has no attribute 'rank'

Answer 1

As of now (July 2021) you can't explain multi-label. The output must be a 1-dimensional vector (having rank 1).

The documentation says it in different places, e.g.:

class Deep(Explainer):  

     def __init__(self, model, data, session=None, learning_phase_flags=None):   

    """ An explainer object for a differentiable model using a given background dataset....

     model : if framework == 'tensorflow', (input : [tf.Tensor], output : tf.Tensor)
             A pair of TensorFlow tensors (or a list and a tensor) that specifies the input and
            output of the model to be explained. Note that SHAP values are specific to a single
            output value, so the output tf.Tensor should be a single dimensional output (,1).

Deep then imported as a DeepExplainer in the package namespace.