Issues with building the LLVM 10 C++ toolchain

Question

Specifically, I'd like to build Clang, lld, lldb, libc++, and compiler-rt, and configure Clang to use these by default. I'm on an x86-64 Linux distro (Oracle Linux), so I believe this should be supported.

This is what I did, after several tries and lots of Google / documentation-reading:

git clone https://github.com/llvm/llvm-project.git -b llvmorg-10.0.0
cd llvm-project

mkdir build
cd build

cmake \
    -D CMAKE_BUILD_TYPE=Release \
    -D CMAKE_C_FLAGS_RELEASE='-DNDEBUG -Ofast' \
    -D CMAKE_CXX_FLAGS_RELEASE='-DNDEBUG -Ofast' \
    -D CMAKE_INSTALL_PREFIX=/opt/llvm10-toolchain \
    -D LLVM_ENABLE_WARNINGS=Off \
    -D LLVM_TARGETS_TO_BUILD=X86 \
    -D LLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;lldb" \
    -D LLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" \
    -D LLVM_INCLUDE_TOOLS=On \
    -D LLVM_BUILD_TOOLS=On \
    -D LLVM_INCLUDE_BENCHMARKS=Off \
    -D LLVM_INSTALL_BINUTILS_SYMLINKS=On \
    -D LLVM_INSTALL_CCTOOLS_SYMLINKS=On \
    -D LLVM_APPEND_VC_REV=On \
    -D LLVM_ENABLE_THREADS=On \
    -D LLVM_ENABLE_LTO=On \
    -D LLVM_ENABLE_BINDINGS=Off \
    -D LLVM_LIBDIR_SUFFIX=64 \
    -D LLVM_BUILD_32_BITS=Off \
    -D LLVM_BUILD_LLVM_DYLIB=Off \
    -D LLVM_INSTALL_TOOLCHAIN_ONLY=On \
    -D CLANG_DEFAULT_LINKER=lld \
    -D CLANG_DEFAULT_CXX_STDLIB=libc++ \
    -D CLANG_DEFAULT_RTLIB=compiler-rt \
    -D LIBCXX_USE_COMPILER_RT=On \
    -D LIBCXXABI_USE_COMPILER_RT=On \
    -D LIBCXXABI_USE_LLVM_UNWINDER=On \
    -G Ninja \
    ../llvm

cmake --build .
cmake --build . --target install

I'm not using bootstrapping, as I have a fairly recent GCC installed and active (GCC 9.3.0) in the environment.

While build succeeds with this, I end up with a non-functional Clang:

> clang++ -v -std=c++17 -Wall -Wextra now.cpp -o now
clang version 10.0.0 (https://github.com/llvm/llvm-project.git d32170dbd5b0d54436537b6b75beaf44324e0c28)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/llvm10-toolchain/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.2
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
 "/opt/llvm10-toolchain/bin/clang-10" -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name now.cpp -mrelocation-model static -mthread-model posix -mframe-pointer=all -fmath-errno -fno-rounding-math -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -dwarf-column-info -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /opt/llvm10-toolchain/lib64/clang/10.0.0 -internal-isystem /opt/llvm10-toolchain/bin/../include/c++/v1 -internal-isystem /usr/local/include -internal-isystem /opt/llvm10-toolchain/lib64/clang/10.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -Wall -Wextra -std=c++17 -fdeprecated-macro -fdebug-compilation-dir /home/gabor -ferror-limit 19 -fmessage-length 0 -fgnuc-version=4.2.1 -fobjc-runtime=gcc -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -faddrsig -o /tmp/now-139cc8.o -x c++ now.cpp
clang -cc1 version 10.0.0 based upon LLVM 10.0.0 default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /opt/llvm10-toolchain/bin/../include/c++/v1
 /usr/local/include
 /opt/llvm10-toolchain/lib64/clang/10.0.0/include
 /usr/include
End of search list.
 "/opt/llvm10-toolchain/bin/ld.lld" --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o now /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crti.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64 -L/opt/llvm10-toolchain/bin/../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../.. -L/opt/llvm10-toolchain/bin/../lib -L/lib -L/usr/lib /tmp/now-139cc8.o -lc++ -lm /opt/llvm10-toolchain/lib64/clang/10.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lc /opt/llvm10-toolchain/lib64/clang/10.0.0/lib/linux/libclang_rt.builtins-x86_64.a /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtend.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crtn.o
ld.lld: error: unable to find library -lc++
ld.lld: error: cannot open /opt/llvm10-toolchain/lib64/clang/10.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such file or directory
ld.lld: error: cannot open /opt/llvm10-toolchain/lib64/clang/10.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such file or directory
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)

For readability, here are the linker search directories mentioned above, in order:

• /opt/llvm10-toolchain/lib64
• /lib64
• /usr/lib64
• /usr/lib
• /opt/llvm10-toolchain/lib
• /lib
• /usr/lib

However, libc++ exists in a directory it doesn't search:

> find /opt/llvm10-toolchain -type f -name 'libc++*'
/opt/llvm10-toolchain/lib64/x86_64-unknown-linux-gnu/c++/libc++.so.1.0
/opt/llvm10-toolchain/lib64/x86_64-unknown-linux-gnu/c++/libc++.so
/opt/llvm10-toolchain/lib64/x86_64-unknown-linux-gnu/c++/libc++.a
/opt/llvm10-toolchain/lib64/x86_64-unknown-linux-gnu/c++/libc++abi.so.1.0
/opt/llvm10-toolchain/lib64/x86_64-unknown-linux-gnu/c++/libc++abi.a

As for libclang_rt.builtins-x86_64.a, something similar does exist, but in a different path:

> find /opt/llvm10-toolchain -type f -name 'libclang_rt*'
/opt/llvm10-toolchain/lib/clang/10.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.builtins.a

I have also tried moving compiler-rt, libcxx, etc. from LLVM_ENABLE_RUNTIMES to LLVM_ENABLE_PROJECTS, but if I do this, compilation fails with an error message while building libc++abi:

/tmp/libc++abi.so.1.0.5xZLnU.ltrans0.ltrans.o::DW.ref.gcc_personality_v0: error: undefined reference to 'gcc_personality_v0'

Even though before it, during CMake configuration, it appears to find it:

-- Looking for gcc_personality_v0 in gcc_s
-- Looking for gcc_personality_v0 in gcc_s - found

I have also tried the same with LLVM 10.0.1-rc1, but it made no difference.

Answer 1

Looks like something in the configuration is off. I'm by far no LLVM/clang expert, so here are my uneducated guesses:

• It's not picking up the correct GCC (you mention you have a recent 9.3.0 GCC installed and active while compiling LLVM, but while actually running clang++ it picks up a GCC in version 4.8.5). Be sure to have the same GCC in your environment while running clang++ as you had while compiling it.
• You're passing LLVM_INSTALL_TOOLCHAIN_ONLY=On as an option. While looking at this site, it seems that the files needed by your clang++ call shouldn't even be installed (though that patch is rather old).
• The differences between the paths to the libclang_rt*s might be caused by LLVM_LIBDIR_SUFFIX=64, though the installation process should handle this case correctly.

I'd recommend building and installing LLVM again and having a closer look at the output. Second, ensure the environments while building and while executing are the same (if you're using environment modules, which I can not recommend enough when using multiple different toolchains, you can define dependencies for this case). After all it's really hard to tell without the build logs and insight into your environment, as I said: these are just some uneducated guesses.