Can I generate a BUILD file in Bazel?

Question

I want to use the go lint tool to generate a BUILD file for bazel.

I would have a go binary that would perform something like this bash script:

#!/bin/bash

cat <<EOF > BUILD
# THIS FILE IS AUTOGENERATED

package(default_visibility = ["//visibility:public"])

load("//go:def.bzl", "go_prefix", "go_library", "go_binary", "go_test")
EOF

for pkg in `go list -f {{.ImportPath}} ./...`; do
    go list -f "`cat test.in`" $pkg >> "BUILD";
done

buildifier -mode fix BUILD

The test.in file contains:

{{ if eq .Name "main" }}
go_binary
{{else}}
go_library
{{end}}
("{{.Name}}",
    srcs=[
        {{range .GoFiles}}
            "{{.}}",
        {{end}} 
    ],
    deps=[
        {{range .Imports}}
            "{{.}}",
        {{end}} 
    ],
    csrcs=[
        {{range .CgoFiles}}
            "{{.}}",
        {{end}} 
    ],
    swig_srcs=[
        {{range .SwigFiles}}
            "{{.}}",
        {{end}} 
    ],
    cxxswig=[
        {{range .SwigCXXFiles}}
            "{{.}}",
        {{end}} 
    ],
    cflags=[
        {{range .CgoCFLAGS}}
            "{{.}}",
        {{end}} 
    ],
    cxxflags=[
        {{range .CgoCXXFLAGS}}
            "{{.}}",
        {{end}} 
    ],
    cppflags=[
        {{range .CgoCPPFLAGS}}
            "{{.}}",
        {{end}} 
    ],           
    {{ with $ctx := context}}
        {{$ctx.InstallSuffix}}
    {{end}}    
)

And this generates:

# THIS FILE IS AUTOGENERATED

package(default_visibility = ["//visibility:public"])

load("//go:def.bzl", "go_prefix", "go_library", "go_binary", "go_test")

go_library

(
    "tensorflow",
    srcs = [
        "doc.go",
        "gen.go",
        "graph.go",
        "session.go",
    ],
    deps = [
        "C",
        "encoding/binary",
        "fmt",
        "github.com/golang/protobuf/proto",
        "github.com/tensorflow/tensorflow/tensorflow/contrib/go/proto",
        "math",
        "reflect",
        "runtime",
        "strings",
        "unsafe",
    ],
    csrcs = [
        "lib.go",
        "tensor.go",
    ],
    swig_srcs = [
    ],
    cxxswig = [
        "tensorflow.swigcxx",
    ],
    cflags = [
        "-I/Users/fmilo/workspace/gopath/src/github.com/tensorflow/tensorflow/tensorflow/contrib/go/../../../",
    ],
    cxxflags = [
        "-I/Users/fmilo/workspace/gopath/src/github.com/tensorflow/tensorflow/tensorflow/contrib/go/../../../",
        "-std=c++11",
    ],
    cppflags = [
    ],
)

go_library

(
    "tensorflow",
    srcs = [
        "allocation_description.pb.go",
        "attr_value.pb.go",
        "config.pb.go",
        "cost_graph.pb.go",
        "device_attributes.pb.go",
        "event.pb.go",
        "function.pb.go",
        "graph.pb.go",
        "kernel_def.pb.go",
        "log_memory.pb.go",
        "master.pb.go",
        "memmapped_file_system.pb.go",
        "meta_graph.pb.go",
        "named_tensor.pb.go",
        "op_def.pb.go",
        "queue_runner.pb.go",
        "saved_tensor_slice.pb.go",
        "saver.pb.go",
        "step_stats.pb.go",
        "summary.pb.go",
        "tensor.pb.go",
        "tensor_description.pb.go",
        "tensor_shape.pb.go",
        "tensor_slice.pb.go",
        "tensorflow_server.pb.go",
        "test_log.pb.go",
        "tf_ops_def.go",
        "types.pb.go",
        "variable.pb.go",
        "versions.pb.go",
        "worker.pb.go",
    ],
    deps = [
        "fmt",
        "github.com/golang/protobuf/proto",
        "github.com/golang/protobuf/ptypes/any",
        "math",
    ],
    csrcs = [
    ],
    swig_srcs = [
    ],
    cxxswig = [
    ],
    cflags = [
    ],
    cxxflags = [
    ],
    cppflags = [
    ],
)

go_library

(
    "tensorflow_error",
    srcs = [
        "error_codes.pb.go",
    ],
    deps = [
        "fmt",
        "github.com/golang/protobuf/proto",
        "math",
    ],
    csrcs = [
    ],
    swig_srcs = [
    ],
    cxxswig = [
    ],
    cflags = [
    ],
    cxxflags = [
    ],
    cppflags = [
    ],
)

go_library

(
    "tensorflow_grpc",
    srcs = [
        "master_service.pb.go",
        "worker_service.pb.go",
    ],
    deps = [
        "fmt",
        "github.com/golang/protobuf/proto",
        "math",
        "tensorflow/core/protobuf",
    ],
    csrcs = [
    ],
    swig_srcs = [
    ],
    cxxswig = [
    ],
    cflags = [
    ],
    cxxflags = [
    ],
    cppflags = [
    ],
)

of course the above BUILD file does not work yet, but I want to make sure first that the approach that I am pursuing is valid.

• How can I specify a pre-BUILD phase in bazel ?
• is it possible?

Answer 1

In general, you cannot generate BUILD files, they must be there at the beginning of the build. Bazel's general build phases are:

• Parse the targets you specified on the command line.
• Look up BUILD files in each directory those specify.
• Parse BUILD files that those targets mention (and so on).
• Generate a tree of actions to execute (e.g., "Run shell script").
• Actually execute the actions.

So, by the time your script is executed, the time for reading BUILD files is long over.

Also, outputs are put in their own directory (under bazel-out) so they wouldn't even be present in your source tree without munging.

That said...

Skylark remote repositories do let you execute code to write files, including BUILD files, in other repositories. So you could, theoretically, have something like (untested):

def _gen_repo_impl(repo_ctx):
  # Writes external/reponame/BUILD
  repo_ctx.execute("/your/bash/script.sh")
  for path in repo_ctx.attr.paths:
    repo_ctx.symlink(path, path.basename)

generated_repo = repository_rule(
    implementation = _gen_repo_impl,
    attrs = {"paths": attr.label_list(allow_files = True)}
)

Then in your WORKSPACE file, do something like:

load('//path/to/your:genrepo.bzl', 'generated_repo')
generated_repo(
    name = "pwd",
    paths = ["src", "test", "third_party"],
)

Then you could build/depend on @pwd//... and the repository rule would generate an external repo with symlinks to src, test, and third_party (theoretically... my code's probably got some typos).