Combining existing rootfs with custom toolchain

Question

I've got a Raspberry PI with Emdebian installed on it, and want to cross-compile projects.

There is plenty of documentation on how to obtain a toolchain and build a simple project with it. I myself managed to build a toolchain with crosstool-ng and wrote a hello world program which works fine.

What I don't get is how to handle cross-compiling more complex projects like Qt, which have dependencies on other libraries. Let's use libdbus as an example, as that is one of Qt's dependencies.

The installed Emdebian already contains libdbus.so, so naturally I'd prefer to use that, instead of cross-compiling my own libdbus.so, as compiling all of Qt's dependencies would take a lot of time.

For cross-compiling, there are two important directories, as far as I understand:

1. The "staging" directory, where all the installed libraries and applications live. This initially is a copy of the toolchain's sysroot directory, and gets populated with more libraries as they are cross-compiled.
2. The "rootfs" directory, which is equivalent to what is on the device - essentially a copy of the staging directory without unneeded stuff like documentation and header files. As far as I understand it, the best approach is to copy required files from the staging directory into the rootfs.

Getting the rootfs directory is easy, as that can be a NFS mount from the device. But how do I get a staging directory for the existing Emdebian installation on the PI? The staging directory needs to include things like dbus headers, which are not installed on the rootfs.

Some people simply install the dbus headers on the device, with apt-get install libdbus-dev, and then use the rootfs as the staging directory. With this setup, there is no distinction between rootfs and staging anymore, with the disadvantage that the rootfs is polluted with headers, documentation and so on. The advantage of course is that it is easy.

What is the best way to get the dbus headers into my staging directory on my host machine? What is the usual approach people use in this situation?

As a side question, why does the approach of obtaining a toolchain, compiling a program and then copying that on a target work at all? The toolchain ships its own versions of libc, libstdc++ etc, are they not incompatible with the versions that are installed on the target? Especially when creating using a custom toolchain compiled with crosstool-ng?

(Note that I am not asking how to compile Qt, I can figure that out myself. My question is more general, about the approach to take when combining a custom toolchain with an existing installation/rootfs)

Answer 1

In my experience, you don't need to compile your dbus. You can do it as

Create Debian cross rootfs by debootstrap by https://wiki.debian.org/EmDebian/CrossDebootstrap
Create your cros-compile toolchain by crosstool-ng, and make sure the kernel version and eglibc version are the same as rootfs created by 1st step
Build QT by

CPPFLAGS=-I<rootfs>/usr/include \
LDFLAGS=-L<rootfs>/lib -L<rootfs>/usr/lib -Wl,-rpath-link,<rootfs>/lib,<rootfs>/usr/lib \
./configure <your options>
make

Install QT into the stage directory by

make install DESTDIR=<stage directory>

Copy QT dependent libraries from rootfs the your stage directory

So you can see that, the stage directory is kept minimum without pollution.