How dynamic linking works, its usage and how and why you would make a dylib

Question

I have read several posts on stack overflow and read about dynamic linking online. And this is what I have taken away from all those readings -

Dynamic linking is an optimization technique that was employed to take full advantage of the virtual memory of the system. One process can share its pages with other processes. For example the libc++ needs to be linked with all C++ programs but instead of copying over the executable to every process, it can be linked dynamically with many processes via shared virtual pages.

However this leads me to the following questions

1. When a C++ program is compiled. It needs to have references to the C++ library functions and code (say for example the code of the thread library). How does the compiler make the executable have these references? Does this not result in a circular dependency between the compiler and the operating system? Since the compiler has to make a reference to the dynamic library in the executable.
2. How and when would you use a dynamic library? How do you make one? What is the specific compiling command that is used to produce such a file from a standard *.cpp file?
3. Usually when I install a library, there is a lib/ directory with *.a files and *.dylib (on mac-OSX) files. How do I know which ones to link to statically as I would with a regular *.o file and which ones are supposed to be dynamically linked with? I am assuming the *.dylib files are dynamic libraries. Which compiler flag would one use to link to these?
4. What are the -L and -l flags for? What does it mean to specify for example a -lusb flag on the command line?

If you feel like this question is asking too many things at once, please let me know. I would be completely ok with splitting this question up into multiple ones. I just ask them together because I feel like the answer to one question leads to another.

Answer 1

When a C++ program is compiled. It needs to have references to the C++ library functions and code (say for example the code for the library).

Assume we have a hypothetical shared library called libdyno.so. You'll eventually be able to peek inside it using using objdump or nm.

objdump --syms libdyno.so

You can do this today on your system with any shared library. objdump on a MAC is called gobjdump and comes with brew in the binutils package. Try this on a mac...

gobjdump --syms /usr/lib/libz.dylib

You can now see that the symbols are contained in the shared object. When you link with the shared object you typically use something like

g++ -Wall -g -pedantic -ldyno DynoLib_main.cpp -o dyno_main

Note the -ldyno in that command. This is telling the compiler (really the linker ld) to look for a shared object file called libdyno.so wherever it normally looks for them. Once it finds that object it can then find the symbols it needs. There's no circular dependency because you the developer asked for the dynamic library to be loaded by specifying the -l flag.

How and when would you use a dynamic library? How do you make one? As in what is the specific compiling command that is used to produce such a file from a standard .cpp file

Create a file called DynoLib.cpp

#include "DynoLib.h"
DynamicLib::DynamicLib() {}
int DynamicLib::square(int a) {
  return a * a;
}

Create a file called DynoLib.h

#ifndef DYNOLIB_H
#define DYNOLIB_H
class DynamicLib {
  public:
  DynamicLib();
  int square(int a); 
};
#endif

Compile them to be a shared library as follows. This is linux specific...

g++ -Wall -g -pedantic -shared -std=c++11 DynoLib.cpp -o libdyno.so

You can now inspect this object using the command I gave earlier ie

objdump --syms libdyno.so

Now create a file called DynoLib_main.cpp that will be linked with libdyno.so and use the function we just defined in it.

#include "DynoLib.h"    
#include <iostream>     
using namespace std;
int main(void) {
  DynamicLib *lib = new DynamicLib();
  std::cout << "Square " << lib->square(1729) << std::endl;
  return 1;
}

Compile it as follows

g++ -Wall -g -pedantic -L. -ldyno DynoLib_main.cpp -o dyno_main
./dyno_main
Square 2989441

You can also have a look at the main binary using nm. In the following I'm seeing if there is anything with the string square in it ie is the symbol I need from libdyno.so in any way referenced in my binary.

nm dyno_runner |grep square
U _ZN10DynamicLib6squareEi

The answer is yes. The uppercase U means undefined but this is the symbol name for our square method in the DynamicLib Class that we created earlier. The odd looking name is due to name mangling which is it's own topic.

How do I know which ones to link to statically as I would with a regular .o file and which ones are supposed to be dynamically linked with?

You don't need to know. You specify what you want to link with and let the compiler (and linker etc) do the work. Note the -l flag names the library and the -L tells it where to look. There's a decent write up on how the compiler finds thing here

gcc Linkage option -L: Alternative ways how to specify the path to the dynamic library

Or have a look at man ld.

What are the -L and -l flags for? What does it mean to specify for example a -lusb flag on the command line?

See the above link. This is from man ld..

-L searchdir

Add path searchdir to the list of paths that ld will search for archive libraries and ld control scripts. You may use this option any number of times. The directories are searched in the order in which they are specified on the command line. Directories specified on the command line are searched before the default directories. All -L options apply to all -l options, regardless of the order in which the options appear. -L options do not affect how ld searches for a linker script unless -T option is specified.`

If you managed to get here it pays dividends to learn about the linker ie ld. It plays an important job and is the source of a ton of confusion because most people start out dealing with a compiler and think that compiler == linker and this is not true.