Menu
I'm trying to run a test program to check if my Anaconda environment is configured correctly. However, when I run my test program I get this error message when the program is setting up graph (on_train_end() callback to be precise):
OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
I'm running the test program on my MacBook Pro 15" 2015 where it is installed macOS Mojave 10.14.1. The Anaconda distribution that I have currently installed is https://repo.anaconda.com/archive/Anaconda2-5.3.0-MacOSX-x86_64.sh.
Here is the test program:
#!/usr/bin/env python
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow import keras
Xs = np.array([
[0, 0],
[0, 1],
[1, 1],
[1, 0]
])
Ys = np.array([
[0],
[1],
[0],
[1]
])
class MyCallback(keras.callbacks.Callback):
def __init__(self):
super(MyCallback, self).__init__()
self.stats = []
def on_epoch_end(self, epoch, logs=None):
self.stats.append({
'loss': logs['loss'],
'acc': logs['acc'],
'epoch': epoch
})
def on_train_end(self, logs=None):
loss_x = []
loss_y = []
acc_x = []
acc_y = []
for e in self.stats:
loss_x.append(e['epoch'])
loss_y.append(e['loss'])
acc_x.append(e['epoch'])
acc_y.append(e['acc'])
plt.plot(loss_x, loss_y, 'r', label='Loss')
plt.plot(acc_x, acc_y, 'b', label='Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Loss / Accuracy')
plt.legend(loc='upper left')
plt.show()
with tf.Session() as session:
model = keras.models.Sequential()
model.add(keras.layers.Dense(10, activation=keras.activations.elu, input_dim=2))
model.add(keras.layers.Dense(1, activation=keras.activations.sigmoid))
model.compile(optimizer=keras.optimizers.Adam(lr=0.05),
loss=keras.losses.mean_squared_error,
metrics=['accuracy'])
model.fit(x=Xs, y=Ys, batch_size=4, epochs=50, callbacks=[MyCallback()])
print("Training complete")
loss, acc = model.evaluate(Xs, Ys)
print(f"loss: {loss} - acc: {acc}")
predictions = model.predict(Xs)
print("predictions")
print(predictions)
I already tried to fix the issue referencing to the answer of this related question. Thus, adding the following lines of code after the import section:
import os
os.environ['KMP_DUPLICATE_LIB_OK'] = 'True'
What I get is another error message, this is the full stack trace:
2018-12-06 10:18:34.262 python[19319:371282] -[NSApplication _setup:]: unrecognized selector sent to instance 0x7ff2b07a3d00
2018-12-06 10:18:34.266 python[19319:371282] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSApplication _setup:]: unrecognized selector sent to instance 0x7ff2b07a3d00'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff2ccf0e65 __exceptionPreprocess + 256
1 libobjc.A.dylib 0x00007fff58d47720 objc_exception_throw + 48
2 CoreFoundation 0x00007fff2cd6e22d -[NSObject(NSObject) __retain_OA] + 0
3 CoreFoundation 0x00007fff2cc92820 ___forwarding___ + 1486
4 CoreFoundation 0x00007fff2cc921c8 _CF_forwarding_prep_0 + 120
5 libtk8.6.dylib 0x0000000b36aeb31d TkpInit + 413
6 libtk8.6.dylib 0x0000000b36a4317e Initialize + 2622
7 _tkinter.cpython-36m-darwin.so 0x0000000b3686ba16 _tkinter_create + 1174
8 python 0x000000010571c088 _PyCFunction_FastCallDict + 200
9 python 0x00000001057f2f4f call_function + 143
10 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
11 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
12 python 0x00000001057f3b1c _PyFunction_FastCallDict + 364
13 python 0x000000010569a8b0 _PyObject_FastCallDict + 320
14 python 0x00000001056c1fe8 method_call + 136
15 python 0x00000001056a1efe PyObject_Call + 62
16 python 0x0000000105743385 slot_tp_init + 117
17 python 0x00000001057478c1 type_call + 241
18 python 0x000000010569a821 _PyObject_FastCallDict + 177
19 python 0x00000001056a2a67 _PyObject_FastCallKeywords + 327
20 python 0x00000001057f3048 call_function + 392
21 python 0x00000001057f0b6f _PyEval_EvalFrameDefault + 47023
22 python 0x00000001057f330c fast_function + 188
23 python 0x00000001057f2fac call_function + 236
24 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
25 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
26 python 0x00000001057f3b1c _PyFunction_FastCallDict + 364
27 python 0x000000010569a8b0 _PyObject_FastCallDict + 320
28 python 0x00000001056c1fe8 method_call + 136
29 python 0x00000001056a1efe PyObject_Call + 62
30 python 0x00000001057f0cc0 _PyEval_EvalFrameDefault + 47360
31 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
32 python 0x00000001057f33ba fast_function + 362
33 python 0x00000001057f2fac call_function + 236
34 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
35 python 0x00000001057f330c fast_function + 188
36 python 0x00000001057f2fac call_function + 236
37 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
38 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
39 python 0x00000001057f33ba fast_function + 362
40 python 0x00000001057f2fac call_function + 236
41 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
42 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
43 python 0x00000001057f33ba fast_function + 362
44 python 0x00000001057f2fac call_function + 236
45 python 0x00000001057f0b6f _PyEval_EvalFrameDefault + 47023
46 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
47 python 0x00000001057f33ba fast_function + 362
48 python 0x00000001057f2fac call_function + 236
49 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
50 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
51 python 0x00000001057f33ba fast_function + 362
52 python 0x00000001057f2fac call_function + 236
53 python 0x00000001057f0abf _PyEval_EvalFrameDefault + 46847
54 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
55 python 0x00000001057f33ba fast_function + 362
56 python 0x00000001057f2fac call_function + 236
57 python 0x00000001057f0b6f _PyEval_EvalFrameDefault + 47023
58 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
59 python 0x00000001057f33ba fast_function + 362
60 python 0x00000001057f2fac call_function + 236
61 python 0x00000001057f0b6f _PyEval_EvalFrameDefault + 47023
62 python 0x00000001057e4209 _PyEval_EvalCodeWithName + 425
63 python 0x000000010583cd4c PyRun_FileExFlags + 252
64 python 0x000000010583c224 PyRun_SimpleFileExFlags + 372
65 python 0x0000000105862d66 Py_Main + 3734
66 python 0x0000000105692929 main + 313
67 libdyld.dylib 0x00007fff59e1608d start + 1
68 ??? 0x0000000000000002 0x0 + 2
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Here is a list of the related dependencies installed in the environment (not related dependencies are omitted for brevity):
Name | Version Build
--------------------|----------------|----------------------
_tflow_select | 2.3.0 | mkl
blas | 1.0 | mkl
intel-openmp | 2019.1 | 144
matplotlib | 3.0.1 | py36h54f8f79_0
mkl | 2018.0.3 | 1
mkl_fft | 1.0.6 | py36hb8a8100_0
mkl_random | 1.0.1 | py36h5d10147_1
numpy | 1.15.4 | py36h6a91979_0
numpy-base | 1.15.4 | py36h8a80b8c_0
tensorboard | 1.12.0 | py36hdc36e2c_0
tensorflow | 1.12.0 | mkl_py36h2b2bbaf_0
tensorflow-base | 1.12.0 | mkl_py36h70e0e9a_0
625
In most cases, this solves the problem:
conda install nomkl
I have tried the following solutions that I have come across. Unfortunately, many of them did not work out and the reasons behind them are also not very clear:
I was using conda installed Tensorflow 2.0 MKL with python3.6 in mac OS Mojave
To downgrade matplotlib. What does it have to do something with OpenMP? Reason not clear but it did not work out.
conda install matplotlib==2.2.3
Allow duplication of OpenMP library because of multiple copies of it exist. This works out but in the warning log, it says this is a workaround and silently produce incorrect results. So, definitely, this is not the way to go, therefore still a proper solution/fix is required.
import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'
To install nomkl. I guess this is to not use MKL based binaries for all the libraries (scipy, numpy, tensorflow etc) but then I do not get the point why to use Tensorflow-MKL? because the whole point is to use MKL binaries to take advantage of the Intel architecture to do fast processing (AVX2 instructions etc). Most of the people said this worked out for them, however, this did not work out for me:
conda install nomkl
Update MKL. It did not work out.
conda install -c intel mkl
Uninstall OpenMP and install it again. It did not work out.
conda uninstall openmp
conda install openmp
Finally, what I did is to uninstall conda installed tensorflow (tf-mkl) and install it again via pip. This has worked out!!! I think this is a proper solution. So, it might mean Intel TF-MKL binaries are broken for macOS. I have an observation this is common for Intel and macOS since other libraries like OpenVINO, pyrealsense2 etc are also not working well in macOS.
conda uninstall tensorflow
pip install tensorflow==2.0.0
Some useful links:
34
I had a similar experience and solutions posted elsewhere were not fixing things for me. Eventually I got unblocked by downgrading my version of matplotlib, i.e. conda install matplotlib=2.2.3
39
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
