libreadline.so.7: undefined symbol: UP

Question

I'm having trouble building cURL and Git from sources when using GNU's Readline 7. When configuring libraries like cURL the result is:

$ ./configure ...
...
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
gawk: symbol lookup error: /usr/local/lib64/libreadline.so.7: undefined symbol: UP
config.status: error: could not create Makefile
Failed to configure cURL

Sure enough:

$ nm -D /usr/local/lib64/libreadline.so | egrep 'UP|DOWN|LEFT|RIGHT'
    U UP

And:

$ nm /usr/local/lib64/libreadline.a | egrep 'UP|DOWN|LEFT|RIGHT'
    U UP

Now, the weird thing is, the messages gawk: symbol lookup error: /usr/local/lib64/libreadline.so.7: undefined symbol: UP and undefined symbol: UP do not show up in cURL's config.log. Its almost as if an Autotools command is producing it, and not the configure test scripts.

Readline's CHANGE log and README do not discuss the change or the missing symbol. I found several posts where other users experience the same problem, but the fix usually degenerates to, downgrade to readline 6.3.

At this point I am not sure if this is a Readline problem; or a problem in a program or library like cURL or Git. Because the symbol is undefined in Readline, I think its closer to a Readline problem (or a library Readline depends on).

My first question is, where should I begin looking for the problem?

My second question is, how should we fix it?

---

Here's the relevant portion of cURL's config.log. There is no tail and no test because cURL does not seem to test for it or it was not logged. Even the error strings are missing from the log.

$ cat curl-7.56.0/config.log
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

  $ ./configure --prefix=/usr/local --libdir=/usr/local/lib64 --enable-shared --
enable-static --enable-optimize --enable-symbol-hiding --enable-http --enable-ft
p --enable-file --enable-ldap --enable-ldaps --enable-rtsp --enable-proxy --enab
le-dict --enable-telnet --enable-tftp --enable-pop3 --enable-imap --enable-smb -
-enable-smtp --enable-gopher --enable-cookies --enable-ipv6 --with-zlib=/usr/loc
al --with-ssl=/usr/local --without-gnutls --without-polarssl --without-mbedtls -
-without-cyassl --without-nss --without-libssh2 --with-libidn2=/usr/local --with
-nghttp2 --with-ca-path=/etc/ssl/certs/ --with-ca-bundle=/etc/ssl/certs/ca-bundl
e.crt

...

---

For completeness, I did not build awk, gawk or mawk from sources and install them somewhere. gawk is Fedora 26's standard one, and it does not use libraries in /usr/local/lib64:

Build-Scripts$ command -v gawk
/bin/gawk

Build-Scripts$ ldd /bin/gawk
    linux-vdso.so.1 (0x00007ffc33d2d000)
    libsigsegv.so.2 => /lib64/libsigsegv.so.2 (0x00007f3da9135000)
    libreadline.so.7 => /lib64/libreadline.so.7 (0x00007f3da8ee9000)
    libmpfr.so.4 => /lib64/libmpfr.so.4 (0x00007f3da8c87000)
    libgmp.so.10 => /lib64/libgmp.so.10 (0x00007f3da8a10000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f3da880c000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f3da84f6000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f3da8125000)
    libtinfo.so.6 => /lib64/libtinfo.so.6 (0x00007f3da7ef9000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f3da95e0000)

Answer 1

What's UP?

Symbol UP is defined in lib_termcap.c and lib_termcap.o in libraries libncurses.so and libncurses.a in package ncurses

Package readline requires curses. ncurses is "New curses".

$ nm -D lib/libncurses.so | grep -w UP
000000000025f6c8 B UP

You must use --with-shared to compile the shared object libraries.

./configure --prefix=/u01/sw --with-shared
make

Answer 2

It appears there were two problems. First, the dynamic linker was selecting the wrong library. That is, the runtime linker incorrectly guessed /usr/bin/gawk was authorized to use /usr/local/lib64/libreadline.so. In my case, /usr/local is full of updates and it is somewhat of a playground. /usr programs should only use libraries from /lib64 and /usr/lib64; and /usr/local programs can use libraries from both /lib64, /usr/lib64 and /usr/local/lib64. They should never mix but we have no way to express the policy to the linker.

The dynamic linker vulnerability has festered for years, and Linux is the only major OS I am aware that still has not solved it. OS X has install names, Windows has manifest, the BSD's have rpath's set at build time ...

Second, the makefiles needed patching. @AnttiHaapala and @JohnBollinger made the observations in the comments but I was struggling with a "it should work" mindset since Red Hat and Fedora are major distros. It appears things are simply untested and the results are a break.

To clear the issue with Fedora this should be done after unpacking the tarball and configuring the library:

for mfile in $(find "$PWD" -name 'Makefile'); do
    sed -i 's|SHLIB_LIBS =|SHLIB_LIBS = -ltinfo|g' "$mfile"
done

After things started working as expected it occurred to me Fedora must be doing the same thing but I did not see it in Fedora's readline.spec. However it was done in readline-7.0-shlib.patch, which is the second patch the spec file applies.

Answer 3

It appears that, in recent ncurses releases, the UP (and related) symbols were moved to libtinfo, which ncurses does not build by default. If you pass the --with-termlib flag to ncurses (./configure --with-shared --with-termlib) it will build this library and the symbols should resolve. Notably, it is important to not follow the suggestion from the previous answer from jww, since that patch will explicitly avoid linking against libtinfo.