Building a simple (hello-world-esque) example of using ld's option -rpath with $ORIGIN

Question

Note: Full working example now below. Original question follows:

I'm having problems using ld's -rpath parameter with $ORIGIN.
As I couldn't find a complete example, I thought I'd try to write one myself, so that I and others can use it later. Once I get it working I'll tidy it up.

I asked about this before, but I think my post was a bit confusing.

The example project builds one shared library and one executable that links to said library.
It's very small (3 files, 22 lines incl buildscript).
You can download the project from here

---

File structure (before building):

• project/
  • src/
    • foo.cpp
    • main.cpp
  • make.sh

---

project/src/foo.cpp

int foo()
  { return 3; }

project/src/main.cpp

int foo();

#include <iostream>
int main()
  {
    std::cout << foo() << std::endl;
    return 0;
  }

project/make.sh

# Make directories:
mkdir -p -v obj
mkdir -p -v lib
mkdir -p -v run

# Build the library:
g++ -c -o obj/foo.o src/foo.cpp -fPIC
g++ -shared -o lib/foo.sh obj/foo.o

# Build the executable:
g++ -c -o obj/main.o src/main.cpp
g++ -o run/main.run obj/main.o -Wl,-rpath,'$ORIGIN/../../lib' -Llib -l:foo.sh

---

From the project directory, run make.sh (make sure it's executable).

---

File structure (after building):

• project/
  • src/
    • foo.cpp
    • main.cpp
  • obj/
    • foo.o
    • main.o
  • lib/
    • foo.so
  • run/
    • main.run
  • make.sh

---

run/main.run should now load lib/foo.sh on execution, from anywhere.

Problems

Currently, this only partly works.
The files compile and link OK, but it fails to link when run from any directory except project (which is the point of the exercise).

Inspecting main.run with readelf -d shows:
0x0000000000000001 (NEEDED) Shared library: [lib/foo.sh]
0x000000000000000f (RPATH) Library rpath: [$ORIGIN/../../lib] Which looks close (I'd rather have [foo.sh] than [lib/foo.sh] but I'll fix that later).

AFAICT the $ORIGIN in -Wl,-rpath,'$ORIGIN/../../lib' means project/run/main.run so this rpath should become project/lib.

I have tried $ORIGIN/.., $ORIGIN/../lib, $ORIGIN/../.., $ORIGIN/../../lib to no avail.

Note: I'm using -l: which requires the complete library filename (amongst other reasons, it's easier to script with variables when all functions take the same name format).

Does anyone know why this isn't working?
Or alternately, does anyone have or know of a complete working example?

Answer 1

(I'd rather have [foo.sh] than [lib/foo.sh] but I'll fix that later).

There's most of your problem: the / in the name stops the dynamic linker from doing the rpath magic.

(Your rpath is wrong too. Think about it: from the shell, if you were currently in the directory where your executable is, how would you get to the directory where your library is? Here, you'd need to cd ../lib. So your rpath should be $ORIGIN/../lib.)

If you built your object as libfoo.so and linked with -Llib -lfoo, the linker would work out what you were intending, and do the right thing. But if you're going to use unusual naming conventions, you'll have to help it out:

1. Change the link line for the library to explicitly set the SONAME for your library to just foo.sh:

   g++ -shared -Wl,-soname,foo.sh -o lib/foo.sh obj/foo.o

2. Fix the rpath:

   g++ -o run/main.run obj/main.o -Wl,-rpath,'$ORIGIN/../lib' -Llib -l:foo.sh

It's useful to run ldd main/main.run to see what's going on. In your original failing case, you'll see something like:

    lib/foo.sh (0xNNNNNNNN)

(the lack of any => /some/resolved/path showing that it's not done any path resolution). In the fixed case, you'll see something like:

    foo.sh => /your/path/to/run/../lib/foo.sh (0xNNNNNNNN)

Answer 2

This is a example of relative-path linking (with ld) by using $ORIGIN in an rpath.

rpath is a path (or set of paths) embedded in binary files (shared libraries (.so) and executables).
These paths are the foremost search paths for the shared libraries the binary must be linked with at runtime.

$ORIGIN is a potential start directory for an rpath path. It resolves to the directory containing the executing file. (eg: $ORIGIN/lib)

The example project builds one shared library and one executable that links to said library using rpath and $ORIGIN.
You can download the project from here.

---

File structure (before building):

• project/
  • src/
    • foo.cpp
    • main.cpp
  • make.sh

---

project/src/foo.cpp

int foo()
  { return 3; }

project/src/main.cpp

int foo();

#include <iostream>
int main()
  {
    std::cout << foo() << std::endl;
    return 0;
  }

project/make.sh

# Make directories:
mkdir -p -v obj
mkdir -p -v lib/dir
mkdir -p -v run

# Build the library:
g++ -c -o obj/foo.o src/foo.cpp -fPIC
g++ -shared -o lib/dir/foo.so -Wl,-soname,foo.so obj/foo.o

# Build the executable:
g++ -c -o obj/main.o src/main.cpp
g++ -o run/main.run obj/main.o -Wl,-rpath,'$ORIGIN/../lib/dir' -Llib/dir -l:foo.so

---

From the project directory, run make.sh (if it won't run, ensure make.sh has execute permissions).

If all went OK, main.run should now load lib/dir/foo.so on execution, regardless of the absolute path to project (you can move it to anywhere), and regardless of the current working directory (you can run it from anywhere).

---

Notes:

• -fPIC instructs the compiler to build relocatable object files (object files build into shared libraries must be relocatable).
• -Wl,-soname,<NAME> embeds <NAME> into the generated library. This should match the name you supply for the -l or -l: options when linking to this library.
• -Wl,-rpath,'<PATH>' embeds <PATH> into the generated library as a runtime library search path (or rpath - see above).
• -L adds a path to the build-time library search path list. (Note: rpath is irrelevant at build-time, -L is irrelevant at runtime).
• -l: adds the filename (without path) of a library to link against. (Similar to -l, except -l: requires the complete filename.

---

File structure (after building):

• project/
  • src/
    • foo.cpp
    • main.cpp
  • obj/
    • foo.o
    • main.o
  • lib/
    • dir/
      • foo.so
  • run/
    • main.run
  • make.sh

---

Note: I'm using -l: which requires the complete library filename (amongst other reasons, it's easier to script with variables when all functions take the same name format).
It is more common to use -l, whereby -l<NAME> denotes lib.so.

Limitations

As far as I'm aware (correct me if I'm wrong) there's no way to add a library inside a subdirectory within a search path (except for adding that directory as a sub-path). This is true for both build-time (-L) and run-time (-rpath) search paths.

So if you have two libraries with the same name but different locations, you won't be able to link them both. (I hope I'm wrong or that this gets fixed).