How to adjust symbols-file in a debian package for a shared lib

Question

I am trying to upgrade ktorrent in my Ubuntu PPA to the latest upstream version. It also requires an updated libktorrent package. It seems libktorrent was changed in incompatible ways and thus results in a new package libktorrent5 instead of the previously available libktorrent4.

However when I try to build the package on my PPA, I get errors about different symbols. I tried some ways to fix it, but it fails with different output each time.

Is there some guide how to correctly generate the symbols-file?

Full build and build-log are here

dh_strip debug symbol extraction: disabling for PPA build dh_strip debug symbol extraction: not doing anything since NO_PKG_MANGLE is given    dh_makeshlibs -Xusr/lib/kde4/ -a -O--parallel -O--
-O--dbg-package=libktorrent-dbg dpkg-gensymbols: warning: some new symbols appeared in the symbols file: see diff output below dpkg-gensymbols: warning: some symbols or patterns disappeared in the symbols file: see diff output below dpkg-gensymbols: warning: debian/libktorrent5/DEBIAN/symbols doesn't match completely debian/libktorrent5.symbols
--- debian/libktorrent5.symbols (libktorrent5_1.3.0-0ubuntu0~ppa4_amd64)
+++ dpkg-gensymbolsNTCQU9   2012-09-30 02:21:19.000000000 +0000 @@ -2912,13 +2912,20 @@   _ZTVN3utp9UTPServer7PrivateE@Base 1.2.0   _ZTVN3utp9UTPServerE@Base 1.2.0   _ZTVN3utp9UTPSocketE@Base 1.2.0
- _ZThn12_N2bt5UTPex5visitE14QSharedPointerINS_4PeerEE@Base 1.3.0
- _ZThn52_N3dht11FindNodeRspD0Ev@Base 1.3.0
- _ZThn52_N3dht11FindNodeRspD1Ev@Base 1.3.0
- _ZThn52_N3dht11GetPeersRspD0Ev@Base 1.3.0
- _ZThn52_N3dht11GetPeersRspD1Ev@Base 1.3.0
- _ZThn8_N2bt4Peer12chunkAllowedEj@Base 1.3.0
- _ZThn8_N2bt4Peer12handlePacketEPKhj@Base 1.3.0
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn12_N2bt5UTPex5visitE14QSharedPointerINS_4PeerEE@Base 1.3.0
+ _ZThn16_N2bt4Peer12chunkAllowedEj@Base 1.3.0-0ubuntu0~ppa4
+ _ZThn16_N2bt4Peer12handlePacketEPKhj@Base 1.3.0-0ubuntu0~ppa4
+ _ZThn24_N2bt5UTPex5visitE14QSharedPointerINS_4PeerEE@Base 1.3.0-0ubuntu0~ppa4
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn52_N3dht11FindNodeRspD0Ev@Base 1.3.0
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn52_N3dht11FindNodeRspD1Ev@Base 1.3.0
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn52_N3dht11GetPeersRspD0Ev@Base 1.3.0
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn52_N3dht11GetPeersRspD1Ev@Base 1.3.0
+ _ZThn80_N3dht11FindNodeRspD0Ev@Base 1.3.0-0ubuntu0~ppa4
+ _ZThn80_N3dht11FindNodeRspD1Ev@Base 1.3.0-0ubuntu0~ppa4
+ _ZThn80_N3dht11GetPeersRspD0Ev@Base 1.3.0-0ubuntu0~ppa4
+ _ZThn80_N3dht11GetPeersRspD1Ev@Base 1.3.0-0ubuntu0~ppa4
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn8_N2bt4Peer12chunkAllowedEj@Base 1.3.0
+#MISSING: 1.3.0-0ubuntu0~ppa4# _ZThn8_N2bt4Peer12handlePacketEPKhj@Base 1.3.0   (c++)"non-virtual thunk to bt::ChunkDownload::getStats(bt::ChunkDownloadInterface::Stats&)@Base"
1.2.0   (c++)"non-virtual thunk to bt::ChunkDownload::~ChunkDownload()@Base" 1.2.0   (c++)"non-virtual thunk to bt::DataCheckerJob::acquired()@Base" 1.2.0 dh_makeshlibs: dpkg-gensymbols -plibktorrent5 -Idebian/libktorrent5.symbols
-Pdebian/libktorrent5 -edebian/libktorrent5/usr/lib/libktorrent.so.5.0.0  returned exit code 1

Answer 1

if a library is released (as different versions) with varying exported symbols, any "consumer" of the library (that is, each application/library that dynamically links against it) needs to track which specific version of the library it needs.

e.g. if you app uses only the symbol "dht::FindNodeRsp::FindNodeRsp()" and this symbol was introduced with version 0.7 of the library, then your app needs at least version 0.7 of the library (even if the current version is 0.9)

on debian, this process is (or: can be) automated with the help of the "symbols" file. e.g. if you package your app, the packaging process (to be precise: dpkg-shlibdeps) will check which symbols your app imports from which library, and cross-check with the library-package's symbol-file to get an estimate of the needed library-version.

it seems your package already has a symbols-file(good!)

the only thing you have to do is update this file, based on the information you already have. you should definitely do this manually, by comparing the diff-output in the build-log to the actual symbols file and add appropriate lines to your symbols file.

example (adding a new symbol)

e.g. imagine your build-log contains a line like the following:

+ _ZThn52_N3dht11FindNodeRspD1Ev@Base 1.3.0

this means that a new symbol _ZThn52_N3dht11FindNodeRspD1Ev@Base has been added (+) in version 1.3.0 of the library. since the mangled name is hardly readable (at least by humans), you might want to run it through c++filt to de-mangle it:

 $ echo _ZThn52_N3dht11FindNodeRspD1Ev@Base | c++filt 
 non-virtual thunk to dht::FindNodeRsp::~FindNodeRsp()@Base

in this case, you should add a new line to your symbols file:

 (c++)"non-virtual thunk to dht::FindNodeRsp::~FindNodeRsp()@Base" 1.3.0

note that you probably should strip away local-version suffixes, e.g. 1.3.0-0ubuntu0~ppa4 should become 1.3.0-0ubuntu0~ (leave the trailing tilde in place)

example (removing a symbol)

unfortunately the output you get indicates, that a number of symbols have been removed from your library. this is bad, as it breaks compatibility! (if your application (only) uses the symbol dht::FindNodeRsp::~FindNodeRsp() which was introduced in version 1.0.3 but removed in version 2.3.0, you can link it against version 1.0.3, but you could also use version 1.2.5, as it still provides this symbol; your app won't care for other symbols that might be present or not in the library).

since it breaks compatibility, the first thing you should make sure is, to bump the major versionnumber of your library-packagename.

e.g. if your library has removed a symbol between (upstream) release "1.0.3" and "1.0.4", and the original (binary) package name was "libfoo1" (used to package upstream "1.0.3"), then you should change the (binary) package name to "libfoo2" (even though upstream version is still 1.something)

this is a MUST, dictated by Debian policy!

renaming the binary package name, most likely involves renaming the symbols file from debian/libfoo1.symbols to debian/libfoo2.symbols

once you have done that, remove the offending lines from the symbols file.

architecture issues: 32bit vs 64bit

you might experience problems with some types that "look" differently on various architectures, notably 64bit systems vs 32bit systems. the most common of these types is size_t, which can expand to either unsigned long or unsigned int, depending on your architecture.

luckily there exist tools to handle (some of) these cases, e.g. the pkgkde-gensymbols replacement for dpkg-gensymbols, which is comes with pkg-kde-tools

to use it, modify the relevant entries in your symbols-file to something like the following (assuming your library exports a symbol "foo::bar(size_t n)"

  (c++|subst)"foo::bar({c++:size_t})@Base" 1.2.3

you then have to tell your debian/rules file to use pkgkde-gensymbols instead of the standard. if you are using CDBS for packaging, add the following line right after the ordinary includes:

  include /usr/share/pkg-kde-tools/makefiles/1/cdbs/symbolshelper.mk

consult the README.Debian of pkg-kde-tools for info on how to do it for other build-systems, e.g. dh and/or the "standard" simple KDE-packages.