Bazel: copy multiple files to binary directory

Question

I need to copy some files to binary directory while preserving their names. What I've got so far:

filegroup(
    name = "resources",
    srcs = glob(["resources/*.*"]),
)

genrule(
    name = "copy_resources",
    srcs = ["//some/package:resources"],
    outs = [ ],
    cmd = "cp $(SRCS) $(@D)",
    local = 1,
    output_to_bindir = 1,
)

Now I have to specify file names in outs but I can't seem to figure out how to resolve the labels to obtain the actual file names.

Answer 1

To make a filegroup available to a binary (executed using bazel run) or to a test (when executed using bazel test) then one usually lists the filegroup as part of the data of the binary, like so:

cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
    data = [
        "//your_project/other/deeply/nested/resources:other_test_files",
    ],
)
# known to work at least as of bazel version 0.22.0

Usually the above is sufficient.

However, the executable must then recurse through the directory structure "other/deeply/nested/resources/" in order to find the files from the indicated filegroup.

In other words, when populating the runfiles of an executable, bazel preserves the directory nesting that spans from the WORKSPACE root to all the packages enclosing the given filegroup.

Sometimes, this preserved directory nesting is undesirable.

---

THE CHALLENGE:

In my case, I had several filegroups located at various points in my project directory tree, and I wanted all the individual files of those groups to end up side-by-side in the runfiles collection of the test binary that would consume them.

---

My attempts to do this with a genrule were unsuccessful.

In order to copy individual files from multiple filegroups, preserving the basename of each file but flattening the output directory, it was necessary to create a custom rule in a bzl bazel extension.

Thankfully, the custom rule is fairly straightforward.

It uses cp in a shell command much like the unfinished genrule listed in the original question.

The extension file:

# contents of a file you create named: copy_filegroups.bzl
# known to work in bazel version 0.22.0
def _copy_filegroup_impl(ctx):
    all_input_files = [
        f for t in ctx.attr.targeted_filegroups for f in t.files
    ]

    all_outputs = []
    for f in all_input_files:
        out = ctx.actions.declare_file(f.basename)
        all_outputs += [out]
        ctx.actions.run_shell(
            outputs=[out],
            inputs=depset([f]),
            arguments=[f.path, out.path],
            # This is what we're all about here. Just a simple 'cp' command.
            # Copy the input to CWD/f.basename, where CWD is the package where
            # the copy_filegroups_to_this_package rule is invoked.
            # (To be clear, the files aren't copied right to where your BUILD
            # file sits in source control. They are copied to the 'shadow tree'
            # parallel location under `bazel info bazel-bin`)
            command="cp $1 $2")

    # Small sanity check
    if len(all_input_files) != len(all_outputs):
        fail("Output count should be 1-to-1 with input count.")

    return [
        DefaultInfo(
            files=depset(all_outputs),
            runfiles=ctx.runfiles(files=all_outputs))
    ]


copy_filegroups_to_this_package = rule(
    implementation=_copy_filegroup_impl,
    attrs={
        "targeted_filegroups": attr.label_list(),
    },
)

Using it:

# inside the BUILD file of your exe
load(
    "//your_project:copy_filegroups.bzl",
    "copy_filegroups_to_this_package",
)

copy_filegroups_to_this_package(
    name = "other_files_unnested",
    # you can list more than one filegroup:
    targeted_filegroups = ["//your_project/other/deeply/nested/library:other_test_files"],
)

cc_binary(
    name = "hello-world",
    srcs = ["hello-world.cc"],
    data = [
        ":other_files_unnested",
    ],
)

You can clone a complete working example here.