My target is dynamically linked against libraries from brew. How to bundle for distribution?

Question

I built an audio plugin. I target .app and .component.

I dynamically link against a brew-installed library, libfluidsynth.

I copied libfluidsynth into the .app / .component.
I used install_name_tool to re-link the binary to point to the bundled libfluidsynth.

libfluidsynth depends on glib, gthread, intl.
I copied those libraries into the bundle, re-linked libfluidsynth to prefer the bundled copy.
I also did the same for those libraries and their dependencies.

Here's a quick peek at what that looks like:

ls /Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks
    libfluidsynth.1.7.1.dylib
    libglib-2.0.0.dylib
    libgthread-2.0.0.dylib
    libintl.8.dylib
    libpcre.1.dylib

otool -L \
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/MacOS/juicysfplugin \
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks/* \
| grep -vE '\t(/System/Library|/usr/lib)'

/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/MacOS/juicysfplugin:
    @executable_path/../Frameworks/libfluidsynth.1.7.1.dylib (compatibility version 1.0.0, current version 1.7.1)
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks/libfluidsynth.1.7.1.dylib:
    @loader_path/../Frameworks/libfluidsynth.1.dylib (compatibility version 1.0.0, current version 1.7.1)
    @loader_path/../Frameworks/libgthread-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks/libglib-2.0.0.dylib:
    @loader_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libpcre.1.dylib (compatibility version 4.0.0, current version 4.9.0)
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks/libgthread-2.0.0.dylib:
    @loader_path/../Frameworks/libgthread-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libpcre.1.dylib (compatibility version 4.0.0, current version 4.9.0)
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks/libintl.8.dylib:
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/git/juicysfplugin/Builds/MacOSX/build/Release/juicysfplugin.app/Contents/Frameworks/libpcre.1.dylib:
    @loader_path/../Frameworks/libpcre.1.dylib (compatibility version 4.0.0, current version 4.9.0)

This worked perfectly for .app. Here, look at the file opens (opensnoop | grep 'dylib'):

EXECNAME      PATH
juicysfplugin  juicysfplugin.app/Contents/MacOS/../Frameworks/libfluidsynth.1.7.1.dylib
juicysfplugin  juicysfplugin.app/Contents/MacOS/../Frameworks/../Frameworks/libgthread-2.0.0.dylib
juicysfplugin  juicysfplugin.app/Contents/MacOS/../Frameworks/../Frameworks/libglib-2.0.0.dylib
juicysfplugin  juicysfplugin.app/Contents/MacOS/../Frameworks/../Frameworks/libintl.8.dylib
juicysfplugin  juicysfplugin.app/Contents/MacOS/../Frameworks/../Frameworks/../Frameworks/libpcre.1.dylib

The .app only looks for dylibs inside its bundled Frameworks folder. Perfect.

Then I did the same copies and re-links upon my .component target. This did not work.

I load the .component into an audio plugin host, and check the file opens:

EXECNAME     PATH
Plugin Host  /Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/MacOS/juicysfplugin
Plugin Host  /usr/local/lib/libfluidsynth.1.7.1.dylib
Plugin Host  /usr/local/opt/glib/lib/libgthread-2.0.0.dylib
Plugin Host  /usr/local/opt/glib/lib/libglib-2.0.0.dylib
Plugin Host  /usr/local/opt/gettext/lib/libintl.8.dylib
Plugin Host  /usr/local/opt/pcre/lib/libpcre.1.dylib

It's looking for libraries under /usr/local. Why? As a sanity-check, I used otool to confirm that I had indeed linked correctly:

otool -L \
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/MacOS/juicysfplugin \
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/Frameworks/* \
| grep -vE '\t(/System/Library|/usr/lib)'

/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/MacOS/juicysfplugin:
    @executable_path/../Frameworks/libfluidsynth.1.7.1.dylib (compatibility version 1.0.0, current version 1.7.1)
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/Frameworks/libfluidsynth.1.7.1.dylib:
    @loader_path/../Frameworks/libfluidsynth.1.dylib (compatibility version 1.0.0, current version 1.7.1)
    @loader_path/../Frameworks/libgthread-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/Frameworks/libglib-2.0.0.dylib:
    @loader_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libpcre.1.dylib (compatibility version 4.0.0, current version 4.9.0)
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/Frameworks/libgthread-2.0.0.dylib:
    @loader_path/../Frameworks/libgthread-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 5401.0.0, current version 5401.3.0)
    @loader_path/../Frameworks/libpcre.1.dylib (compatibility version 4.0.0, current version 4.9.0)
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/Frameworks/libintl.8.dylib:
    @loader_path/../Frameworks/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0)
/Users/me/Library/Audio/Plug-Ins/Components/juicysfplugin.component/Contents/Frameworks/libpcre.1.dylib:
    @loader_path/../Frameworks/libpcre.1.dylib (compatibility version 4.0.0, current version 4.9.0)

I thought I linked it correctly. I used exactly the same script (it's automated and parameterised). What am I doing wrong? How did the audio plugin host know to lookup dependencies under /usr/local? Why were my library links ignored?

I've released the binaries to: https://github.com/Birch-san/juicysfplugin/releases/tag/1.0.1

My relinking script is here: https://github.com/Birch-san/juicysfplugin/blob/master/Builds/MacOSX/relink-build-for-distribution.sh

The main difference between how the .app and .component targets are used is:

• .app is standalone
• .component is an Audio Unit plugin, that you load into a DAW / audio plugin host.

So, perhaps the runtime dependency resolution is different when a parent process is responsible for loading our executable?

Answer 1

Okay, I got it working.

macOS load-time linking is documented at man dyld.

The problem is that I told my binary to look for libraries relative to @executable_path.

This works fine for .app, because the .app's binary is the executable.

But for my .vst and .component plugins, the binary is loaded into a different executable: the audio plugin host.

So, if we want to lookup a library relative to our binary (juicysfplugin.component/Contents/MacOS/juicysfplugin), we need to use @loader_path, not @executable_path.

---

Now, onto the other mysteries… why did the load-time linker ignore my install path (@executable_path/../Frameworks/libfluidsynth.1.7.1.dylib), and instead look for find fluidsynth under /usr/local/lib/libfluidsynth.1.7.1.dylib?

It's because of DYLD_FALLBACK_LIBRARY_PATH!

It is used as the default location for libraries not found in their install path. By default, it is set to $(HOME)/lib:/usr/local/lib:/lib:/usr/lib.

I believe that it failed to find the library in the install path. It then looked for that leaf file name — libfluidsynth.1.7.1.dylib — under a few directories, including /usr/local/lib (which succeded).

Why didn't I see the failed file lookups in opensnoop? Probably it uses a different syscall than open/open_nocancel/open_extended. For example, running stat on a file does not show up in opensnoop.

It's also possible that the load-time linking is done by a dyld process. SIP would disallow attaching DTrace to this process.