Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to run a shell command at analysis time in bazel?

I'm trying to bake the mercurial version into my Bazel file, so that I can get something like this:

# These I set manually, since they're "semantic"
MAJOR_VERSION = 2
MINOR_VERSION = 3
BUGFIX_VERSION = 1

# This should be the result of `hg id -n`
BUILD_VERSION = ?

apple_bundle_version(
    name = "my_version",
    build_version = "{}.{}.{}.{}".format(MAJOR_VERSION, MINOR_VERSION, BUGFIX_VERSION, BUILD_VERSION),
    short_version_string = "{}.{}.{}".format(MAJOR_VERSION, MINOR_VERSION, BUGFIX_VERSION),
)

This obviously is not hermetic, so I know it violates some of Bazel's assumptions, so I'm open to other options.

Here's some possible options:

  1. Actually run hg id -n during Bazel analysis, which I don't know how to do.

  2. Pass the build version in via command line, e.g., --define=build_version=$(hg id -n). Unfortunately, this requires a separate command to wrap bazel build.

  3. Manually set the BUILD_VERSION. Obviously, this would be annoying.

Is there a way to do #1? What are my other options?

like image 616
Jesse Beder Avatar asked Dec 03 '17 19:12

Jesse Beder


People also ask

How do I run Bazel commands?

To run Bazel, go to your base workspace directory or any of its subdirectories and type bazel . % bazel help [Bazel release bazel-<version>] Usage: bazel <command> <options> ... Available commands: analyze-profile Analyzes build profile data.

What is Genrule in Bazel?

A genrule generates one or more files using a user-defined Bash command. Genrules are generic build rules that you can use if there's no specific rule for the task. For example, you could run a Bash one-liner.

How does Bazel run work?

When running a build or a test, Bazel does the following: Loads the BUILD files relevant to the target. Analyzes the inputs and their dependencies, applies the specified build rules, and produces an action graph. Executes the build actions on the inputs until the final build outputs are produced.

What is Bazel command?

Bazel starts exactly one server per specified output base. Typically there is one output base per workspace - however, with this option you may have multiple output bases per workspace and thereby run multiple builds for the same client on the same machine concurrently.


1 Answers

Yes, you can do this with a custom --workspace_status_command and a genrule that processes the information and generates source files with this data.

EDIT: I removed the parts about the --stamp flag, it's not needed.

Summary

  1. Build with --workspace_status_command=/path/to/binary with a custom binary or shell script that runs hg and outputs the information you need.
  2. Write a genrule with stamp=1.

Details

1. --workspace_status_command=/path/to/binary

The --workspace_status_command=<path> flag lets you specify a binary.

Bazel runs this binary before each build. The binary should write key-value pairs to stdout. Bazel partitions the keys into two buckets: "stable" and "volatile". (The names "stable" and "volatile" are a bit counter-intuitive, so don't think much about them.)

Bazel then writes the key-value pairs into two files:

  • bazel-out/stable-status.txt contains all keys and values where the key's name starts with STABLE_
  • bazel-out/volatile-status.txt contains the rest of the keys and their values

The contract is:

  • "stable" keys' values should change rarely, if possible. If the contents of stable-status.txt change, it invalidates the actions that depend on them, e.g. the genrule.cmd if that genrule has stamp=1. In other words, if a stable key's value changes, it'll make Bazel rebuild stamped actions. Therefore the stable status should not contain things like timestamps, because they change all the time, and would make Bazel rebuild the stamped actions with each build.
  • "volatile" keys' values may change often. Bazel expects them to change all the time, like timestamps do, and duly updates the volatile-status.txt file. In order to avoid rebuilding stamped actions all the time though, Bazel pretends that the volatile file never changes. In other words, if the volatile status file is the only one whose contents changed, that will not invalidate actions that depend on it. If other inputs of the actions have changed, then Bazel rebuilds that action, and the action may then use the updated volatile status. But just the volatile status changing alone will not invalidate the action.

Example for my-status.sh:

#!/bin/bash
echo STABLE_GIT_BRANCH $(git rev-parse HEAD)
echo MY_TIMESTAMP $(date)

2. Write a genrule with stamp=1.

This attribute is undocumented, which surprises me. I'll file a bug about that.

Example for foo/BUILD:

genrule(
    name = "x",
    srcs = ["input.txt"],
    outs = ["x.txt"],
    cmd = " ; ".join([
        "( echo 'volatile data:'",
        "cat bazel-out/volatile-status.txt",
        "echo ---",
        "echo 'stable data:'",
        "cat bazel-out/stable-status.txt",
        ") > $@",
    ]),
    stamp = 1,
)

Putting it all together

The genrule isn't rebuilt

...when only bazel-out/volatile-status.txt changes:

  $ bazel build --workspace_status_command=/tmp/foo/ws.sh //foo:x &>/dev/null && cat bazel-genfiles/foo/x.txt
volatile data:
BUILD_TIMESTAMP 1512379211456
MY_TIMESTAMP Mon Dec 4 10:20:11 CET 2017
---
stable data:
BUILD_EMBED_LABEL 
BUILD_HOST <redacted>
BUILD_USER <redacted>
STABLE_GIT_BRANCH d3fed125d00f6f61bfbfe05f4566656cdac1ea6e

  $ cat bazel-out/volatile-status.txt 
BUILD_TIMESTAMP 1512379425898
MY_TIMESTAMP Mon Dec 4 10:23:45 CET 2017

  $ bazel build --workspace_status_command=/tmp/foo/ws.sh //foo:x &>/dev/null && cat bazel-genfiles/foo/x.txt
volatile data:
BUILD_TIMESTAMP 1512379211456
MY_TIMESTAMP Mon Dec 4 10:20:11 CET 2017
---
stable data:
BUILD_EMBED_LABEL 
BUILD_HOST <redacted>
BUILD_USER <redacted>
STABLE_GIT_BRANCH d3fed125d00f6f61bfbfe05f4566656cdac1ea6e

  $ cat bazel-out/volatile-status.txt 
BUILD_TIMESTAMP 1512379441919
MY_TIMESTAMP Mon Dec 4 10:24:01 CET 2017

The genrule is rebuilt

...when the stable status or the genrule's inputs change:

  $ echo bar > foo/input.txt 

  $ bazel build --workspace_status_command=/tmp/foo/ws.sh //foo:x &>/dev/null && cat bazel-genfiles/foo/x.txt
volatile data:
BUILD_TIMESTAMP 1512379566646
MY_TIMESTAMP Mon Dec 4 10:26:06 CET 2017
---
stable data:
BUILD_EMBED_LABEL 
BUILD_HOST <redacted>
BUILD_USER <redacted>
STABLE_GIT_BRANCH d3fed125d00f6f61bfbfe05f4566656cdac1ea6e


  $ git checkout HEAD~1 &>/dev/null

  $ bazel build --workspace_status_command=/tmp/foo/ws.sh //foo:x &>/dev/null && cat bazel-genfiles/foo/x.txt
volatile data:
BUILD_TIMESTAMP 1512379594890
MY_TIMESTAMP Mon Dec 4 10:26:34 CET 2017
---
stable data:
BUILD_EMBED_LABEL 
BUILD_HOST <redacted>
BUILD_USER <redacted>
STABLE_GIT_BRANCH b3da717469e23f5293297175a80709956416fd2c
like image 57
László Avatar answered Sep 23 '22 19:09

László