Library not loaded: @rpath/libmysqlclient.21.dylib Reason: image not found Django migrate error using mysqlclient DB driver and MySQL 8 with macOS

Question

While changing to a MySQL database from the default SQLite one that Django uses by default, I ran into this tricky error while trying to run python manage.py migrate --database mysql:

Traceback (most recent call last):
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 15, in <module>
    import MySQLdb as Database
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/MySQLdb/__init__.py", line 18, in <module>
    import _mysql
ImportError: dlopen(/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/_mysql.cpython-37m-darwin.so, 2): Library not loaded: @rpath/libmysqlclient.21.dylib
  Referenced from: /Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/_mysql.cpython-37m-darwin.so
  Reason: image not found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 20, in <module>
    execute_from_command_line( sys.argv )
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/base.py", line 350, in execute
    self.check()
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/base.py", line 379, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 59, in _run_checks
    issues = run_checks(tags=[Tags.database])
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/checks/registry.py", line 71, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/checks/database.py", line 9, in check_database_backends
    for conn in connections.all():
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 217, in all
    return [self[alias] for alias in self]
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 217, in <listcomp>
    return [self[alias] for alias in self]
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 202, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
    return import_module('%s.base' % backend_name)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 20, in <module>
    ) from err
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?

I received this error after having already installed mysqlclient via pip install mysqlclient and its connector through Homebrew with brew install mysql-connector-c, so this is a bit of a puzzling error. Even after following the connector instructions by modifying the mysql_config file in the connector's bin folder, I still ran into this issue. This does appear to be at least somewhat related to that I'm running macOS given the documentation on this connector's known bug.

Upon inspecting my stack trace further, I noticed the Library not loaded: @rpath/libmysqlclient.21.dylib error up near the top. Shortly after that is another possible clue I found: Reason: image not found, but I'm not quite sure what that's supposed to mean.

I did poke around in /usr/local/Cellar/mysql-connector-c to see if there was anything possibly missing with the connector I installed, and sure enough, the libmysqlclient.21.dylib file was missing. In its place was what I assume is an older version of the file since it was named libmysqlclient.18.dylib. Out of curiosity, I copied the .dylib that my MySQL install came with and moved that copy under the same name to the connector's folder and to /usr/local/lib where I also found another libmysqlclient.18.dylib file. However, my migration still failed, but I did get a slightly different stack trace:

Traceback (most recent call last):
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 15, in <module>
    import MySQLdb as Database
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/MySQLdb/__init__.py", line 18, in <module>
    import _mysql
ImportError: dlopen(/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/_mysql.cpython-37m-darwin.so, 2): Library not loaded: @rpath/libmysqlclient.21.dylib
  Referenced from: /Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/_mysql.cpython-37m-darwin.so
  Reason: no suitable image found.  Did find:
    /usr/local/lib/libmysqlclient.21.dylib: file too short
    /usr/local/lib/libmysqlclient.21.dylib: file too short

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 20, in <module>
    execute_from_command_line( sys.argv )
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/base.py", line 350, in execute
    self.check()
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/base.py", line 379, in check
    include_deployment_checks=include_deployment_checks,
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 59, in _run_checks
    issues = run_checks(tags=[Tags.database])
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/checks/registry.py", line 71, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/core/checks/database.py", line 9, in check_database_backends
    for conn in connections.all():
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 217, in all
    return [self[alias] for alias in self]
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 217, in <listcomp>
    return [self[alias] for alias in self]
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 202, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/utils.py", line 110, in load_backend
    return import_module('%s.base' % backend_name)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/xoadra/Programming/Syntax/Python/Environments/pfitzer/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 20, in <module>
    ) from err
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
Did you install mysqlclient?

This time I get the following error:

Reason: no suitable image found.  Did find:
        /usr/local/lib/libmysqlclient.21.dylib: file too short
        /usr/local/lib/libmysqlclient.21.dylib: file too short

With that, I suspect the problem may have something to do with the .dylib version that my MySQL connector installed with, but I'm not sure how to fix this exactly. I do find it curious that simply copying a file over managed to change my stack trace ever so slightly so I guess I ought to look into that more. Thoughts?

Answer 1

Just create a symbolic link in /usr/local/lib

sudo ln -s /usr/local/mysql/lib/libmysqlclient.21.dylib /usr/local/lib/libmysqlclient.21.dylib

Answer 2

I've recently had this issue when I was trying to install Django and mod_wsgi on my MacBook pro (MacOS Catallina). Setting LD_LIBRARY_PATH, DYLD_LIBRARY_PATH, adding sym link and moving the libmysqlclient.21.dylib to /usr/lib/ didn't work. I had to struggle for a few days but finally I got this solution. The thing is I had to modify the library path in the libmysqlclient.21.dylib. Fortunately we have a tool to do it. The culprit was @rpath/libmysqlclient.21.dylib. It looked like @rpath wasn't working.

Ok you can check the paths referred in a .so file using otool. It comes with Xcode. This is what you get when you run it on the mysql .so file (should be in the MySQLdb directory under your site-packages).

$ otool -L _mysql.cpython-38-darwin.so

_mysql.cpython-38-darwin.so:
    @rpath/libmysqlclient.21.dylib (compatibility version 21.0.0, current version 21.0.0)
    libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
    libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

So we modify the filepath in the _mysql.cpython-38-darwin.so. There is a tool for that. It is install_name_tool. I ran these commands. Basically I changed @rpath to absolute path, and also the libssl and the licrypto paths to absolute path.

$ install_name_tool -change @rpath/libmysqlclient.21.dylib /usr/local/mysql/lib/libmysqlclient.21.dylib _mysql.cpython-38-darwin.so

$ install_name_tool -change libssl.1.1.dylib /usr/local/mysql/lib/libssl.1.1.dylib  _mysql.cpython-38-darwin.so

$ install_name_tool -change libcrypto.1.1.dylib /usr/local/mysql/lib/libcrypto.1.1.dylib _mysql.cpython-38-darwin.so

My Django worked great after these changes.