How to build LLVM (clang,clang++) for Apple M1?

Question

I am trying to build LLVM compilers so that I can enable OpenMP on the Apple M1. I am using the LLVM development tree, (since I saw some OpenMP runtime go into that for this recently).

I have ended up with this script to invoke cmake:

# Xcode, Ninja
BUILD_SYSTEM=Ninja
BUILD_TAG=Ninja

cmake ../llvm \
      -G$BUILD_SYSTEM -B ${BUILD_TAG}_build \
      -DCMAKE_OSX_ARCHITECTURES='arm64' \
      -DCMAKE_C_COMPILER=`which clang` \
      -DCMAKE_CXX_COMPILER=`which clang++` \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_BUILD_WITH_INSTALL_RPATH=1 \
      -DCMAKE_INSTALL_PREFIX=$HOME/software/clang-12.0.0/arm64 \
      -DLLVM_ENABLE_WERROR=FALSE \
      -DLLVM_TARGETS_TO_BUILD='AArch64' \
      -DLLVM_ENABLE_PROJECTS='clang;openmp,polly' \
      -DLLVM_DEFAULT_TARGET_TRIPLE='aarch64-apple-darwin20.1.0'

The compilers used here are

$ /usr/bin/clang --version
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: arm64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

ninja can then successfully build clang, clang++ and the OpenMp runtime and install them. (As simple, Arm64 images targeting Arms64)

$ file ~/software/clang-12.0.0/arm64/bin/clang
/Users/jcownie/software/clang-12.0.0/arm64/bin/clang: Mach-O 64-bit executable arm64
$ ~/software/clang-12.0.0/arm64/bin/clang --version
clang version 12.0.0 (https://github.com/llvm/llvm-project.git 879c15e890b4d25d28ea904e92497f091f796019)
Target: aarch64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Users/jcownie/software/clang-12.0.0/arm64/bin

Which all looks sane, except that when I try to compile anything with them they are missing the include path to get system headers.

$ ~/software/clang-12.0.0/arm64/bin/clang hello.c
hello.c:1:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
         ^~~~~~~~~
1 error generated.

So, after all that,

1. Does anyone know how to fix that include path problem?
2. Does anyone know how to configure and build a fat binary for the compilers (and libraries) so that the x86_64 embedded compiler targets x86_64 and the aarch64 binary aarch64? (This is what the Xcode clang and clang++ do...) My attempt at this ended up with a compiler fat binary where both architectures targeted x86_64 :-(

Thanks

Answer 1

You can set -DDEFAULT_SYSROOT=/path/to/MacOSX11.1.sdk at build time or do export SDKROOT=/path/to/MacOSX11.1.sdk at runtime.

You need to compile with clang -arch arm64 -arch x86_64 to get a fat binary out of clang. You need to do this for Apple clang as well.

Answer 2

UPDATED 8 Feb 2021 Homebrew now supports the M1 based Arm machines, so using that is a better answer than the one below. The info below is potentially still useful if you want to do this on your own, but using brew is likely to be much simpler.

Pre-brew answer

I haven't found a clean solution, but in case it helps anyone else, I do have a horrible hack.

The full recipe, then is configure with this script, then build and install.

# Xcode, Ninja
BUILD_SYSTEM=Ninja
BUILD_TAG=ninja
INSTALLDIR=$HOME/software/clang-12.0.0/arm64

cmake ../llvm \
      -G$BUILD_SYSTEM -B ${BUILD_TAG}_build \
      -DCMAKE_OSX_ARCHITECTURES='arm64' \
      -DCMAKE_C_COMPILER=`which clang` \
      -DCMAKE_CXX_COMPILER=`which clang++` \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX=$INSTALLDIR \
      -DLLVM_LOCAL_RPATH=$INSTALLDIR/lib \
      -DLLVM_ENABLE_WERROR=FALSE \
      -DLLVM_TARGETS_TO_BUILD='AArch64' \
      -DLLVM_DEFAULT_TARGET_TRIPLE='aarch64-apple-darwin20.1.0' \
      -DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)" \
      -DLLVM_ENABLE_PROJECTS='clang;openmp;polly;clang-tools-extra;libcxx;libcxxabi' \
#      -DLLVM_ENABLE_PROJECTS='clang;openmp;polly' 

That gives a compiler that finds the right headers, but won't link successfully if OpenMP is used because it doesn't pass on any useful -L path or add a necessary rpath.

To overcome that I created a small shell script that sits in my ~/bin, at the front of my $PATH, which adds those extra linker flags.

#
# A truly awful hack, but it seems necessary.
# Install this with execute permissions as clang and clang++ in
# a directory early in your path, so that it is executed when clang or
# clang++ is needed.
#
# For brew...
INSTALLDIR=/usr/local/opt/llvm
# For a local build.
INSTALLDIR=${HOME}/software/clang-12.0.0/arm64/

# Find out the name of this file, and then invoke the same file in the
# compiler installation, adding the necessary linker directives
CMD=`echo $0 | sed "s/\/.*\///"`
${INSTALLDIR}/bin/${CMD} -L${INSTALLDIR}/lib -Wl,-rpath,${INSTALLDIR}/lib $*

I am not recommending this particularly; there should clearly be a better way to make it work, but it'll do for now, and lets me get back to using the compiler rather than building it!

Answer 3

I was able to build with -DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)" -DCMAKE_INSTALL_PREFIX=/Users/foo/lokal/ and install into the lokal/bin lokal/lib path. Once that is done you can use LD_LIBRARY_PATH=/Users/foo/lokal/lib and all the libraries should be found without mucking with anything else rpath related.