Bootstrapping Bazel for Raspberry Pi 4 failed, jni_md.h not found

Question

I'm currently trying to compile the latest version of Bazel (2.2.0) from source on a headless Raspberry Pi 4 that has Raspbian Buster installed. I was using this page as an installation guide.

I had no issue following the instructions on that page (all dependencies installed, Java 8 installed, swapsize changed, etc etc) until the point where I started compiling. The build failed with this error:

ERROR: /home/pi/bazel/src/main/java/com/google/devtools/build/lib/syntax/BUILD:150:1: C++ compilation of rule '//src/main/java/com/google/devtools/build/lib/syntax:libcpu_profiler.so' failed (Exit 1): gcc failed: error executing command
  (cd /tmp/bazel_hpBqNseU/out/execroot/io_bazel && \
  exec env - \
    PATH=/home/pi/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games \
    PWD=/proc/self/cwd \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/arm-opt/bin/src/main/java/com/google/devtools/build/lib/syntax/_objs/libcpu_profiler.so/cpu_profiler_unimpl.pic.d '-frandom-seed=bazel-out/arm-opt/bin/src/main/java/com/google/devtools/build/lib/syntax/_objs/libcpu_profiler.so/cpu_profiler_unimpl.pic.o' -fPIC -iquote . -iquote bazel-out/arm-opt/bin -iquote external/bazel_tools -iquote bazel-out/arm-opt/bin/external/bazel_tools -isystem external/bazel_tools/tools/jdk/include -isystem bazel-out/arm-opt/bin/external/bazel_tools/tools/jdk/include -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c src/main/java/com/google/devtools/build/lib/syntax/cpu_profiler_unimpl.cc -o bazel-out/arm-opt/bin/src/main/java/com/google/devtools/build/lib/syntax/_objs/libcpu_profiler.so/cpu_profiler_unimpl.pic.o)
Execution platform: //:default_host_platform
In file included from src/main/java/com/google/devtools/build/lib/syntax/cpu_profiler_unimpl.cc:17:
bazel-out/arm-opt/bin/external/bazel_tools/toolsls/jdk/include/jni.h:45:10: fatal error: jni_md.h: No such file or directory
 #include "jni_md.h"
          ^~~~~~~~~~
compilation terminated.

However, running find / -name jni_md.h 2> /dev/null returns

/usr/lib/jvm/java-8-openjdk-armhf/include/linux/jni_md.h

So it's not as if the header file is missing from my system.

From looking around online (this question specifically) I apparently need to include an additional argument for gcc so it can see the file? However, I'm not sure where exactly I can find the specific build file I would need to edit in order to do so.

If anyone can help point me in the right direction, or if there's another solution that can fix this error, it would be appreciated.

Answer 1

The "easy" first part of your question

From the other question you linked to and the location of the file jni_md.h on your system, here is the "include" command option that you need to add to your gcc command:

gcc -I/usr/lib/jvm/java-8-openjdk-armhf/include/linux

The more tricky second part

The second part of your question is how to modify the build configuration of Bazel so that the appropriate inclusion is made. Big projects that need to compile on many target platforms rely on complex configuration files in which the path to platform-specific files/compilers are specified. It can be overwhelming and confusing at first.

1. Where is the problem?

From looking at your error, I looks like the error comes from file /home/pi/bazel/src/main/java/com/google/devtools/build/lib/syntax/BUILD:150:1

Below is an excerpt taken from GitHub:

This part of the build script shows various configurations depending on the architecture target platform (windows/freebsd ...) as depending on the platform, the location of the file jni_md.h will be different.

# The C++ portion of the Starlark CPU profiler.
cc_binary(
    name = "libcpu_profiler.so",
    srcs = select({
        "//src/conditions:darwin": ["cpu_profiler_posix.cc"],
        "//src/conditions:linux_x86_64": ["cpu_profiler_posix.cc"],
        "//conditions:default": ["cpu_profiler_unimpl.cc"],
    }),
    linkshared = 1,
    deps = [":jni"],
)

# TODO(adonovan): move this to @bazel_tools//tools/jdk:jni where it belongs.
# TODO(adonovan): why is there no condition for "just linux"?
cc_library(
    name = "jni",
    hdrs = ["@bazel_tools//tools/jdk:jni_header"] + select({
        "//src/conditions:linux_x86_64": ["@bazel_tools//tools/jdk:jni_md_header-linux"],
        "//src/conditions:darwin": ["@bazel_tools//tools/jdk:jni_md_header-darwin"],
        "//src/conditions:freebsd": ["@bazel_tools//tools/jdk:jni_md_header-freebsd"],
        "//src/conditions:openbsd": ["@bazel_tools//tools/jdk:jni_md_header-openbsd"],
        "//src/conditions:windows": ["@bazel_tools//tools/jdk:jni_md_header-windows"],
        "//conditions:default": [],
    }),
    includes = ["../../../../../../../../../external/bazel_tools/tools/jdk/include"] + select({
        # Remove these crazy prefixes when we move this rule.
        "//src/conditions:linux_x86_64": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/linux"],
        "//src/conditions:darwin": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/darwin"],
        "//src/conditions:freebsd": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/freebsd"],
        "//src/conditions:openbsd": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/openbsd"],
        "//src/conditions:windows": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/win32"],
        "//conditions:default": [],
    }),
)

From your error:

Execution platform: //:default_host_platform

It looks like you are compiling the project in the "default" configuration. If we look at the BUILD file, we can see that there are no "cc_library" path included when building under the default configuration:

cc_library(
    name = "jni",
    hdrs = ["@bazel_tools//tools/jdk:jni_header"] + select({
    "//conditions:default": [], # IT IS EMPTY
    }),
    includes = ["../../../../../../../../../external/bazel_tools/tools/jdk/include"] + select({
    "//conditions:default": [], # ALSO EMPTY HERE
    }),
)

This explains why the path to the location of the jni_md.h is not added and why you get a compilation error.

2. How to fix the problem?

It looks like there is some discussion between developers to add a simple "linux" case into the configuration. However, you seem to be compiling under the "Default" configuration. This may be the root cause of your problem but I will assume that there is a reason you are using this configuration.

Since your system looks like it is a "linux type, adding the "linux configuration" to the "default" case may fix the problem in your particular case.

In the BUILD file, try to change the cc_library section to:

cc_library(
    name = "jni",
    hdrs = ["@bazel_tools//tools/jdk:jni_header"] + select({
    ...
    "//conditions:default": ["@bazel_tools//tools/jdk:jni_md_header-linux"],
    }),
    includes = ["../../../../../../../../../external/bazel_tools/tools/jdk/include"] + select({
    ...
    "//conditions:default": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/linux"],
    }),
)

---

This is more a "hotfix" than a permanent solution. Let us know if it works or if you are getting a different error! Comments and other suggestions are welcomed. As suggested in another answer, using an older version of Bazel may also remove the problem.

Answer 2

Because I can't post any comments yet, I'm posting an answer. Without the help of Patrick I couldn't do this. My solution is based on Patricks' answer.

I've created a patch file for the above fix, and used condition:arm instead of default.

Save this as bazel-2.2.0-compile-armhf.patch in the bazel build root directory:

--- src/main/java/com/google/devtools/build/lib/syntax/BUILD    2020-03-30 14:42:31.018189718 +0200
+++ src/main/java/com/google/devtools/build/lib/syntax/BUILD.2  2020-03-30 14:42:06.278705784 +0200
@@ -168,6 +168,7 @@
         "//src/conditions:freebsd": ["@bazel_tools//tools/jdk:jni_md_header-freebsd"],
         "//src/conditions:openbsd": ["@bazel_tools//tools/jdk:jni_md_header-openbsd"],
         "//src/conditions:windows": ["@bazel_tools//tools/jdk:jni_md_header-windows"],
+        "//src/conditions:arm": ["@bazel_tools//tools/jdk:jni_md_header-linux"],
         "//conditions:default": [],
     }),
     includes = ["../../../../../../../../../external/bazel_tools/tools/jdk/include"] + select({
@@ -177,6 +178,7 @@
         "//src/conditions:freebsd": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/freebsd"],
         "//src/conditions:openbsd": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/openbsd"],
         "//src/conditions:windows": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/win32"],
+        "//src/conditions:arm": ["../../../../../../../../../external/bazel_tools/tools/jdk/include/linux"],
         "//conditions:default": [],
     }),
 )

Then patch the relevant BUILD file

$ patch -p0  < bazel-2.2.0-compile-armhf.patch
patching file src/main/java/com/google/devtools/build/lib/syntax/BUILD

Now you can compile Bazel 2.2.0 on a Raspberry Pi (or any other armhf device)