OS X linker unable to find symbols from a C file which only contains variables

Question

I am having problems with the linker when porting a C library from Linux (Ubuntu) to OS X. The C code is auto-generated from Matlab, so ideally I don't want to change the code itself.

The problem seems to be in a C file which contains ONLY uninitialised variable declarations, which are then EXTERNed by other C files to implement the Matlab algorithms. The OS X linker is apparently unable to recognise symbols from this file. The same source code works fine on Linux, so I want to understand how the OS X linker is behaving differently, and whether there is a flag I can pass to it to change the behaviour.

The static library builds without errors/warnings. But when building an application which references the static library, the following error message (on OS X) is thrown:

Undefined symbols for architecture x86_64:
  "_my_var", referenced from:
      _algorithm in libtestlibrary.a(algorithm.o)
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status

An 'nm' shows that libtestlibrary.a does indeed contain symbol _my_var.

A much simplified version of the code from Matlab follows below.

Library code:

// DATA.C : declaration of data
#include "data.h"
int my_var;

// DATA.H - extern declaration of data
#ifndef H_DATA
#define H_DATA
extern int my_var;
#endif

// ALGORITHM.C - performs the calculation
#include "data.h"
int algorithm(int x) {
  my_var += x;
  return my_var;
}

//ALGORITHM.H - declaration of library API
#ifndef H_ALGORITHM
#define H_ALGORITHM
int algorithm(int x);
#endif

Library build commands:

gcc -c algorithm.c
gcc -c data.c
ar rcs libtestlibrary.a data.o algorithm.o

Application code:

// MAIN.C : Code which calls into the static library
#include "algorithm.h"
int main() {
  int x = 1;
  x = algorithm(x);
  return 0;
}

Application build commands:

gcc -c main.c
gcc -o testapp main.o -ltestlibrary

If I change the definition in data.c to 'int my_var=0', so that the variable is initialised, then the library and application build correctly on both Linux and OS X. However, as I said above, I don't want to change the code because it is auto-generated from Matlab.

Thanks in advance for your help!

Answer 1

The answer of @Sergey L. is excellent, and will solve the problem. However, it doesn't capture the essence of what's going on. The behavior is actually in the OS X archiver ar, which doesn't list common symbols in the table of contents (TOC) of an archive. I created the files as above, and here's a session with the archiver (OS X 10.9.4, Xcode 5.1.1):

$ gcc -c algorithm.c
$ gcc -c data.c
$ ar rcs libtestlibrary.a data.o algorithm.o
$ ar x libtestlibrary.a '__.SYMDEF SORTED'
$ od -c __.SYMDEF\ SORTED
0000000   \b  \0  \0  \0  \0  \0  \0  \0  \0 002  \0  \0 020  \0  \0  \0
0000020    _   a   l   g   o   r   i   t   h   m  \0  \0  \0  \0  \0 240
0000040

The TOC appears in the archive as an entry named __.SYMDEF SORTED. As you can see the TOC doesn't contain my_var. This is the essence of the problem.

You can add common symbols to the TOC using ranlib -c. Here's how this works:

$ ranlib -c libtestlibrary.a
$ ar x libtestlibrary.a '__.SYMDEF SORTED'
$ od -c __.SYMDEF\ SORTED
0000000  020  \0  \0  \0  \b  \0  \0  \0 020 002  \0  \0  \0  \0  \0  \0
0000020  210  \0  \0  \0 030  \0  \0  \0   _   m   y   _   v   a   r  \0
0000040    _   a   l   g   o   r   i   t   h   m  \0   Ѓ  ** 177  \0  \0
0000060

As you can see, ranlib -c added my_var to the TOC.

Now the link step will work:

$ gcc -L . -o testapp main.o -ltestlibrary
$

Thus, it's not absolutely necessary to recompile the code to get the link to work.

The man page for ranlib says this:

   -c     Include  common symbols as definitions with respect to the table
          of contents.  This is seldom the intended behavior  for  linking
          from  a  library,  as  it forces the linking of a library member
          just because it uses an uninitialized global that  is  undefined
          at  that  point  in  the  linking.  This option is included only
          because this was the original behavior of ranlib.   This  option
          is not the default.

So, this is an intentional deviation from traditional behavior, to avoid linking unneeded modules into the output. I'll probably use -fno-common in my own development.

Answer 2

Your problem is that you do not initialise may_var.

If you do not initialise a data symbol and put it into a static library then it will be created as a common symbol. Those are not recognised my the OS X linker when linking static libraries.

If you initialise it (data.c):

#include "data.h"
int my_var = 0;

Then the compiler will put it into a different section and it will properly link in a static library.

Edit:

Alternatively you can pass the -fno-common option to gcc

gcc -c data.c -fno-common

Then this will instruct gcc not to generate common symbols for uninitialised variables and then you can link them in libraries.

This is an issue with the Mach-O executable format on OS X and is described here.