Incremental linking using gcc on linux. Is it possible?

Question

The way my team's project is developed, we generate a Shared Object library for our application from all all of our .o object files. My task (hopefully it is specific enough but also general enough to be of use to others!) is to link in only the object files that have changed since the last time the executable was created. For example, here is the command line that i use to build the .so:

g++34 -shared -rdynamic -m64 -Wl,-rpath,'$ORIGIN' MyObject1.o MyObject2.o MyObject3.o MyObject4.o   -o libMySharedLibrary.so

Which works as expected! :) My goal is to be able to link in only the changed object files from now on, to speed up the concurrent linking process. An example command would be:

g++34 -shared -rdynamic -m64 -Wl,-rpath,'$ORIGIN' MyObject1.o MyObject3.o   -o libMySharedLibrary.so

Which would update libMySharedLibrary.so with the newer object files, while maintaining the older object files in libMySharedLibrary.so also. In actuality, when I generate libMySharedLibrary.so using the above command, the file-size is much smaller than that of when all object files are included, so I can almost be sure that the above command isn't doing what I want.

Through my research I've found that there is a -i option for the linker which is the same as the -r option, which appears to just combine all the object files into one large object file as well. Unfortunately it does not appear that this is what I want.

In short, I'd like to link in only the changed object files after the initial link, resulting in quicker linking process for the future links. Is there a way to do this?

EDIT: An example of what I've tried with -i/-r:

Example command: g++34 -Wl,-r -nostdlib -rdynamic -m64 -Wl,-rpath,'$ORIGIN' MyObject1.o MyObject2.o MyObject3.o MyObject4.o -o AllMyObjects.o

I had to add the -nostdlib tag to stop it from yelling at me about needing it, and removed -shared because shared objects are not allowed with the -r tag.

This command would appear to slam all of my .o files into one big .o file. So If I could just update that .o file from here on out with only the changed .o files, that would be great. After AllMyObjects.o was initially created, I tried this command: g++34 -Wl,-r -nostdlib -rdynamic -m64 -Wl,-rpath,'$ORIGIN' MyObject1.o MyObject3.o -o AllMyObjects.o, but it would also create a much smaller (file-size-wise) AllMyObjects.o, so I'm assuming it can't possibly have all of the object files. I feel like this is something that I'm likely making a small mistake on. Anyone have any advice? Thanks in advance.

Answer 1

It looks like you're right about -shared and -r not working together. I was skeptical about your old GCC version, but even on Ubuntu 10.10 I can see the same:

$ ld -shared -r
/usr/bin/ld.bfd.real: -r and -shared may not be used together

Unfortunately, that means you've reached a dead-end if you absolutely need shared objects. The binutils linker simply doesn't implement it.

If static libraries are an option for you, they are simply archives that can easily be manipulated with the ar utility.

Otherwise, you'll have to look at different linkers or compiler suites. I can't guarantee that you'll find this feature, though, it seems exotic.

Answer 2

You can sort of get the behavior you're after using archive/static libraries, but the initial link will still take the same amount of time.

Using an archive file:

# Initially create the archive
ar r libmylib.a <all object files>

# Create your shared object (re-use this line after libmylib.a is updated)
g++ -shared -rdynamic -m64 -Wl,-rpath,'$ORIGIN' libmylib.a -o libmylib.so     

# Update the archive file
ar r libmylib.a updated1.o updated2.o

As I said, it will still take the same amount of time to actually link the .so as it did before.