undefined symbol: _ZN10tensorflow8OpKernel11TraceStringEPNS_15OpKernelContextEb

Question

I'm getting the error below:

/usr/bin/python3.6 /home/daya/Deep_Learning/Keras-Tuner/HiddenLayersAndHiddenNeurons.py

2020-12-22 09:42:14.472136: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0

Traceback (most recent call last):
  File "/home/daya/Deep_Learning/Keras-Tuner/HiddenLayersAndHiddenNeurons.py", line 2, in <module>
    from tensorflow import keras
  File "/home/daya/.local/lib/python3.6/site-packages/tensorflow/__init__.py", line 435, in <module>
    _ll.load_library(_main_dir)

  File "/home/daya/.local/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 153, in load_library
    py_tf.TF_LoadLibrary(lib)

tensorflow.python.framework.errors_impl.NotFoundError: /usr/local/lib/python3.6/dist-packages/tensorflow/core/kernels/libtfkernel_sobol_op.so: undefined symbol: _ZN10tensorflow8OpKernel11TraceStringEPNS_15OpKernelContextEb

System config:

Ubuntu 18.04, cuda11.1 cudnn8, GTX 1660, Tensorflow-gpu 2.4

Answer 1

In case this is an issue for you and downgrading Tensorflow is not an option, here is a hack:

delete tensorflow/core/kernels/libtfkernel_sobol_op.so altogether. This operator is a special purpose op in classic computer version for edge extraction and for 99.99% of the use cases of TF (training DNN) you won't need this operator.

Answer 2

Looks like it is the following issue (same symbol _ZN10tensorflow8OpKernel11TraceStringEPNS_15OpKernelContextEb is missing): https://github.com/tensorflow/text/issues/385

Did you try to update Tensorflow?