Failing to run HWASan build on Android: dlopen failed: TLS symbol "(null)"

Question

I've followed the instructions over on https://developer.android.com/ndk/guides/hwasan to enable HWASan for my app:

• I switched to using a shared libc++
• I've added target_compile_options(${TARGET} PUBLIC -fsanitize=hwaddress -fno-omit-frame-pointer) target_link_options(${TARGET} PUBLIC -fsanitize=hwaddress) to the CMakeLists.txt.
• I only build for arm64-v8a
• I added useLegacyPackaging = true
• I added debuggable true
• I added app/src/main/resources/lib/wrap.sh

A few sanity checks to make sure that I enabled HWAsan correctly:

• Without the target_compile_options and target_link_options lines, the app runs properly.
• The apk contains libc++_shared.so, showing I correctly enabled shared libc++
• The apk contains /lib/wrap.sh
• The app uses a single .so: libfoobar.so, and readelf -d libfoobar.so | grep NEEDED prints:

0x0000000000000001 (NEEDED)             Shared library: [libclang_rt.hwasan-aarch64-android.so]
0x0000000000000001 (NEEDED)             Shared library: [libandroid.so]
0x0000000000000001 (NEEDED)             Shared library: [liblog.so]
0x0000000000000001 (NEEDED)             Shared library: [libEGL.so]
0x0000000000000001 (NEEDED)             Shared library: [libOpenSLES.so]
0x0000000000000001 (NEEDED)             Shared library: [libc++_shared.so]
0x0000000000000001 (NEEDED)             Shared library: [libGLESv2.so]
0x0000000000000001 (NEEDED)             Shared library: [libm.so]
0x0000000000000001 (NEEDED)             Shared library: [libdl.so]
0x0000000000000001 (NEEDED)             Shared library: [libc.so]

When I run the apk on my Pixel 7 with Android 14, the following error happens:

dlopen failed: TLS symbol "(null)" in dlopened "/apex/com.android.runtime/lib64/bionic/libclang_rt.hwasan-aarch64-android.so"

Full error:

09-13 21:43:19.312 28028 28028 D nativeloader: Load /data/app/~~VeP7A7hHnvshxaanDdLEyQ==/com.qux-GBK8s7jkWQ4TXb2mglMtUA==/lib/arm64/libfoobar.so using ns clns-4 from class loader (caller=/data/app/~~VeP7A7hHnvshxaanDdLEyQ==/com.qux-GBK8s7jkWQ4TXb2mglMtUA==/base.apk!classes2.dex): dlopen failed: TLS symbol "(null)" in dlopened "/apex/com.android.runtime/lib64/bionic/libclang_rt.hwasan-aarch64-android.so" referenced from "/apex/com.android.runtime/lib64/bionic/libclang_rt.hwasan-aarch64-android.so" using IE access model

My .so does not do any funny business regarding TLS. I disabled all exotic compilation flags. I bumped the minSdk to 29 out of an abundance of caution.

Any clue as to what is going on?

Answer 1

I'm a bit late, but I came across the same issue. Your APK is missing hwasan symbols. The hwasan shared library must be available in the lib directory of your apk. If you extract your apk you want to see:

• your library
• shared c++ library
• hwasan library
• wrap.sh which sets LD_HWASAN=1

If you are using using a custom APK build config in your CMakeLists.txt, you may need to add something to copy this in. An easy way to test the theory is to extract using apktool, copy in the missing library, rebuild, sign, and test.

Answer 2

I got the issue the same to you:

08-12 14:38:49.765  6736  6736 E AndroidRuntime: java.lang.UnsatisfiedLinkError: dlopen failed: TLS symbol "(null)" in dlopened "/apex/com.android.runtime/lib64/bionic/libclang_rt.hwasan-aarch64-android.so" referenced from "/apex/com.android.runtime/lib64/bionic/libclang_rt.hwasan-aarch64-android.so" using IE access model
08-12 14:38:49.765  6736  6736 E AndroidRuntime:        at java.lang.Runtime.loadLibrary0(Runtime.java:1090)
08-12 14:38:49.765  6736  6736 E AndroidRuntime:        at java.lang.Runtime.loadLibrary0(Runtime.java:1012)
08-12 14:38:49.765  6736  6736 E AndroidRuntime:        at java.lang.System.loadLibrary(System.java:1765)
08-12 14:38:49.765  6736  6736 E AndroidRuntime:        at com.seekting.test.cases.CPPSizeActivity.onClick(CPPSizeActivity.kt:8)

and I find the reason，before I fix the issue，My wrap.sh content is here

#!/system/bin/sh
LD_HWASAN=1
exec "$@"

and I check out the NDK file:~Android/sdk/ndk/26.1.10909125/wrap.sh/hwasan.sh file

#!/system/bin/sh
LD_HWASAN=1 exec "$@"

the difference is NDK file is just one line，My wrap.sh is two lines，so you need change your wrap content like the NDK path file. May be it will be helpful for you.