How to integrate pretty-printing as part of build in bazel

Question

Right now, I have a really dumb pretty-print script which does a little git-fu to find files to format (unconditionally) and then runs those through clang-format -i. This approach has several shortcomings:

1. There are certain files which are enormous and take forever to pretty print.
2. The pretty printing is always done, regardless of whether or not the underlying file actually changed or not.

In the past, I was able to do things with CMake that had several nice properties which I would like to reproduce in bazel:

1. Only ever build code after it has gone through linting / pretty printing / etc.
2. Only lint / pretty print / etc. stuff that has changed
3. Pretty print stuff regardless of whether or not it is under VC or not

In CMake-land, I used this strategy, inspired by SCons proxy-target trickery:

1. Introduce a dummy target (e.g. source -> source.formatted). The action associated with this target does two things: a) run clang-format -i source, b) output/touch a file called source.formatted (this guarantees that for reasonable file systems, if source.formatted is newer than source, source doesn't need to be reformatted)

2. Add a dummy target (target_name.aggregated_formatted) which aggregates all the .formatted files corresponding to a particular library / executable target's sources

3. Make library / executable targets depend on target_name.aggregated_formatted as a pre-build step

Any help would be greatly appreciated.

Answer 1

@abergmeier is right. Let's take it one step further by implementing the macro and its components.

We'll use the C++ stage 1 tutorial in bazelbuild/examples.

Let's first mess up hello-world.cc:

#include <ctime>



#include <string>

#include <iostream>

std::string get_greet(const std::string& who) {
      return "Hello " + who;
}

void print_localtime() {
    std::time_t result =
          std::time(nullptr);
  std::cout << std::asctime(std::localtime(&result));
}

int main(int argc, char** argv) {
  std::string who = "world";
  if (argc > 1) {who = argv[1];}
  std::cout << get_greet(who) << std::endl;
  print_localtime();


  return 0;
}

This is the BUILD file:

cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
)

Since cc_binary doesn't know anything about clang-format or linting in general, let's create a macro called clang_formatted_cc_binary and replace cc_binary with it. The BUILD file now looks like this:

load(":clang_format.bzl", "clang_formatted_cc_binary")

clang_formatted_cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
)

Next, create a file called clang_format.bzl with a macro named clang_formatted_cc_binary that's just a wrapper around native.cc_binary:

# In clang_format.bzl
def clang_formatted_cc_binary(**kwargs):
    native.cc_binary(**kwargs)

At this point, you can build the cc_binary target, but it's not running clang-format yet. We'll need to add an intermediary rule to do that in clang_formatted_cc_binary which we'll call clang_format_srcs:

def clang_formatted_cc_binary(name, srcs, **kwargs):
    # Using a filegroup for code cleaniness
    native.filegroup(
        name = name + "_unformatted_srcs",
        srcs = srcs,
    )

    clang_format_srcs(
        name = name + "_formatted_srcs",
        srcs = [name + "_unformatted_srcs"],
    )

    native.cc_binary(
        name = name,
        srcs = [name + "_formatted_srcs"],
        **kwargs
    )

Note that we have replaced the native.cc_binary's sources with the formatted files, but kept the name to allow for in-place replacements of cc_binary -> clang_formatted_cc_binary in BUILD files.

Finally, we'll write the implementation of the clang_format_srcs rule, in the same clang_format.bzl file:

def _clang_format_srcs_impl(ctx):
    formatted_files = []

    for unformatted_file in ctx.files.srcs:
        formatted_file = ctx.actions.declare_file("formatted_" + unformatted_file.basename)
        formatted_files += [formatted_file]
        ctx.actions.run_shell(
            inputs = [unformatted_file],
            outputs = [formatted_file],
            progress_message = "Running clang-format on %s" % unformatted_file.short_path,
            command = "clang-format %s > %s" % (unformatted_file.path, formatted_file.path),
        )

    return struct(files = depset(formatted_files))

clang_format_srcs = rule(
    attrs = {
        "srcs": attr.label_list(allow_files = True),
    },
    implementation = _clang_format_srcs_impl,
)

This rule goes through every file in the target's srcs attribute, declaring a "dummy" output file with the formatted_ prefix, and running clang-format on the unformatted file to produce the dummy output.

Now if you run bazel build :hello-world, Bazel will run the actions in clang_format_srcs before running the cc_binary compilation actions on the formatted files. We can prove this by running bazel build with the --subcommands flag:

$ bazel build //main:hello-world --subcommands
..
SUBCOMMAND: # //main:hello-world_formatted_srcs [action 'Running clang-format on main/hello-world.cc']
.. 
SUBCOMMAND: # //main:hello-world [action 'Compiling main/formatted_hello-world.cc']
.. 
SUBCOMMAND: # //main:hello-world [action 'Linking main/hello-world']
..

Looking at the contents of formatted_hello-world.cc, looks like clang-format did its job:

#include <ctime>
#include <string>

#include <iostream>

std::string get_greet(const std::string& who) { return "Hello " + who; }

void print_localtime() {
  std::time_t result = std::time(nullptr);
  std::cout << std::asctime(std::localtime(&result));
}

int main(int argc, char** argv) {
  std::string who = "world";
  if (argc > 1) {
    who = argv[1];
  }
  std::cout << get_greet(who) << std::endl;
  print_localtime();
  return 0;
}

If all you want are the formatted sources without compiling them, you can run build the target with the _formatted_srcs suffix from clang_format_srcs directly:

$ bazel build //main:hello-world_formatted_srcs
INFO: Analysed target //main:hello-world_formatted_srcs (0 packages loaded).
INFO: Found 1 target...
Target //main:hello-world_formatted_srcs up-to-date:
  bazel-bin/main/formatted_hello-world.cc
INFO: Elapsed time: 0.247s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action