Why am I receive AlreadyExistsError?

Question

When I train my binary classification via keras I received this error:

AlreadyExistsError: Resource __per_step_16/training_4/Adam/gradients/lstm_10/while/ReadVariableOp_8/Enter_grad/ArithmeticOptimizer/AddOpsRewrite_Add/tmp_var/struct tensorflow::TemporaryVariableOp::TmpVar
     [[{{node training_4/Adam/gradients/lstm_10/while/ReadVariableOp_8/Enter_grad/ArithmeticOptimizer/AddOpsRewrite_Add/tmp_var}} = TemporaryVariable[dtype=DT_FLOAT, shape=[64,256], var_name="training_4...dd/tmp_var", _device="/job:localhost/replica:0/task:0/device:CPU:0"](^training_4/Adam/gradients/lstm_10/while/strided_slice_11_grad/StridedSliceGrad)]]

I do the following code:

file = pd.read_csv('train_stemmed.csv')
Y = list(map(int,file['target'].values))
X = list(map(str,file['question_text'].values))

MAXLEN = 100
tokenizer = Tokenizer()
tokenizer.fit_on_texts(X)

X_seq = tokenizer.texts_to_sequences(X)
X_seq_pad = pad_sequences(X_seq, maxlen=MAXLEN)
X_train, X_test, Y_train, Y_test = train_test_split(X_seq_pad, Y, test_size=0.2)
vocab_len = len(tokenizer.word_index) + 1

model = Sequential()
model.add(Embedding(vocab_len, 100, input_length=MAXLEN))
model.add(Conv1D(64, 5, 5, activation='relu'))
model.add(MaxPooling1D(pool_size=5))
model.add(BatchNormalization())
model.add(LSTM(64)) 
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(X_train,
          epochs=2,
          batch_size=128,
          y=Y_train,
          validation_data=(X_test, Y_test),
          verbose=1)

What's wrong?

Answer 1

Adding the following code before the line model = Sequential() will stop this error.

from tensorflow.core.protobuf import rewriter_config_pb2
from tensorflow.keras.backend import set_session
tf.keras.backend.clear_session()  # For easy reset of notebook state.

config_proto = tf.ConfigProto()
off = rewriter_config_pb2.RewriterConfig.OFF
config_proto.graph_options.rewrite_options.arithmetic_optimization = off
session = tf.Session(config=config_proto)
set_session(session)

Answer 2

This is an open issue on tf github (https://github.com/tensorflow/tensorflow/issues/23780) and is related to Grappler Optimization. 2 solutions -

1. You can turn off arithmetic optimizations as per Nandeesh's accepted answer

2. You can reduce memory usage (for eg., decreasing layer(s)/layers' size etc.)