Compile boost as universal library (Intel and Apple Silicon architectures)

Question

I am trying to build boost library as dylib on MacOS. I need to build it for both the Intel architecture and the upcoming Apple Silicon (arm64) architecture.

I downloaded boost and ran the following commands:

./bootstrap.sh
./b2 -address-model=64 architecture=combined -a

lipo -archs always shows produced dylibs architecture is x86_64.

I have Xcode12 beta and MacOS Catalina 10.15.7,

I can build a sample universal library if I create a project in Xcode and set archs arm64 x86_64 in build settings.

Running command ./b2 cxxflags="-arch arm64 -arch x86_64" fails with following errors:

    "clang++" -x c++ -fvisibility-inlines-hidden -m64 -O3 -Wall -fvisibility=hidden -Wno-inline -arch arm64 -arch x86_64 -ftemplate-depth-255 -fvisibility=hidden -fvisibility-inlines-hidden -DBOOST_ALL_NO_LIB=1 -DNDEBUG -I"." -c -o "bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o" "libs/serialization/src/polymorphic_xml_iarchive.cpp"

...failed clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_iarchive.o...
clang-darwin.compile.c++ bin.v2/libs/serialization/build/clang-darwin-12.0/release/link-static/threading-multi/visibility-hidden/polymorphic_xml_oarchive.o
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:32:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from libs/serialization/src/polymorphic_xml_oarchive.cpp:16:
In file included from ./boost/serialization/config.hpp:18:
In file included from ./boost/config.hpp:57:
In file included from ./boost/config/platform/macos.hpp:28:
In file included from ./boost/config/detail/posix_features.hpp:18:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/unistd.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_dev_t;         /* dev_t */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
        ^
note: '__uint128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
        ^

Answer 1

The following works for me on Big Sur using Xcode 12.2. On Catalina 10.15.7 (19H15) I get the same error messages as OP.

./bootstrap.sh --with-libraries=regex,date_time 
./b2 architecture=combined cxxflags="-arch x86_64 -arch arm64"

Answer 2

I'm on Monterey with an M1 (XCode 13.1) and failed to get any of the other answers to work, but I combined stuff from Navan and Petr along with some other bits to get the libboost_coroutine to work on x86_64. I arrived at the following script which builds all the boost libraries (why not, it doesn't take long on the M1):

#!/bin/sh

rm -rf arm64 x86_64 universal stage bin.v2
rm -f b2 project-config*
./bootstrap.sh cxxflags="-arch x86_64 -arch arm64" cflags="-arch x86_64 -arch arm64" linkflags="-arch x86_64 -arch arm64"
./b2 toolset=clang-darwin target-os=darwin architecture=arm abi=aapcs cxxflags="-arch arm64" cflags="-arch arm64" linkflags="-arch arm64" -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang-darwin target-os=darwin architecture=x86 cxxflags="-arch x86_64" cflags="-arch x86_64" linkflags="-arch x86_64" abi=sysv binary-format=mach-o -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do 
  lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib); 
done
for dylib in universal/*; do
  lipo $dylib -info;
done

This script prints out the lipo info for each dylib in the universal directory, so you can see quickly that every one has both x86_64 and arm64 inside. I had the following result, maybe it's useful for you to see if your output matches:

Architectures in the fat file: universal/libboost_atomic.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_chrono.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_container.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_context.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_contract.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_coroutine.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_date_time.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_filesystem.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_graph.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_iostreams.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_locale.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_log.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_log_setup.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_numpy27.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_prg_exec_monitor.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_program_options.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_python27.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_random.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_regex.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_serialization.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_addr2line.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_basic.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_stacktrace_noop.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_system.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_thread.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_timer.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_type_erasure.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_unit_test_framework.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_wave.dylib are: x86_64 arm64 
Architectures in the fat file: universal/libboost_wserialization.dylib are: x86_64 arm64 

Answer 3

After spending considerable amount of time here:

https://github.com/bfgroup/b2/issues/105

I was able to compile boost on arm64. The command line is:

./b2 architecture=arm address-model=64 asmflags=--target=arm64-apple-darwin21.2.0 cflags=--target=arm64-apple-darwin21.2.0 cxxflags=--target=arm64-apple-darwin21.2.0 linkflags=--target=arm64-apple-darwin21.2.0 -s NO_LZMA=1 -s NO_ZSTD=1 abi=aapcs

the LZMA and ZSTD flags are there since I did not have an universal binary for those libs on my machine.

Answer 4

I was facing the same issue and came across this answer, which was for creating universal binaries for i386 and x86_64.

To summarise the answer, you need to run ./b2 twice ( with different toolsets as clang always builds for x86_64, even if you pass the appropriate CXXFlags

This is the script I used to first individually generate the libraries I required and then combined them using lipo

#!/bin/sh

rm -rf arm64 x86_64 universal
./bootstrap.sh --with-toolset=clang --with-libraries=thread,system,filesystem,program_options,serialization 
./b2  cxxflags="-arch arm64" toolset=darwin -a
mkdir -p arm64 && cp stage/lib/*.dylib arm64
./b2 toolset=clang cxxflags="-arch arm64 -arch x86_64" -a
mkdir x86_64 && cp stage/lib/*.dylib x86_64
mkdir universal
for dylib in arm64/*; do 
  lipo -create -arch arm64 $dylib -arch x86_64 x86_64/$(basename $dylib) -output universal/$(basename $dylib); 
done