Nested Bazel Projects

Question

I'm trying to build a project that uses TensorFlow Serving, so I made a directory my_dir with a WORKSPACE file, cloned the serving repo into it, put my custom files into a directory my_project, configured tensorflow within tensorflow_serving, built tensorflow serving from my_dir/serving with

bazel build //tensorflow_serving/...

everything builds fine there, then I try to build a python file imitating mnist_export and put it in my_dir and make a BUILD file

py_binary(
   name = "export_cnn",
   srcs = [
       "export_cnn.py",
   ],
   deps = [
       "@tf//tensorflow:tensorflow_py",
       "@tf_serving//tensorflow_serving/session_bundle:exporter",
   ],
)

However, when I run

bazel build //my_project:export_cnn

I get the following errors:

ERROR: 

.../bazel/_bazel_me/3ef3308a843af155635e839105e8da5c/external/tf/tensorflow/core/BUILD:92:1: null failed: protoc failed: error executing command bazel-out/host/bin/external/tf/google/protobuf/protoc '--cpp_out=bazel-out/local_linux-fastbuild/genfiles/external/tf' -Iexternal/tf -Iexternal/tf/google/protobuf/src ... (remaining 1 argument(s) skipped).
tensorflow/core/framework/step_stats.proto: File not found.
tensorflow/core/framework/device_attributes.proto: File not found.
tensorflow/core/framework/graph.proto: File not found.
tensorflow/core/framework/tensor.proto: File not found.
tensorflow/core/protobuf/config.proto: File not found.
tensorflow/core/protobuf/worker.proto: Import "tensorflow/core/framework/step_stats.proto" was not found or had errors.
tensorflow/core/protobuf/worker.proto: Import "tensorflow/core/framework/device_attributes.proto" was not found or had errors.
tensorflow/core/protobuf/worker.proto: Import "tensorflow/core/framework/graph.proto" was not found or had errors.
tensorflow/core/protobuf/worker.proto: Import "tensorflow/core/framework/tensor.proto" was not found or had errors.
tensorflow/core/protobuf/worker.proto: Import "tensorflow/core/protobuf/config.proto" was not found or had errors.
tensorflow/core/protobuf/worker.proto:41:12: "DeviceAttributes" is not defined.
tensorflow/core/protobuf/worker.proto:64:3: "GraphDef" is not defined.
tensorflow/core/protobuf/worker.proto:72:3: "GraphOptions" is not defined.
tensorflow/core/protobuf/worker.proto:141:3: "TensorProto" is not defined.
tensorflow/core/protobuf/worker.proto:180:3: "StepStats" is not defined.
tensorflow/core/protobuf/worker.proto:225:3: "BusAdjacency" is not defined.
tensorflow/core/protobuf/worker.proto:227:3: "BusAdjacency" is not defined.
tensorflow/core/protobuf/worker.proto:232:3: "TensorProto" is not defined.
tensorflow/core/protobuf/worker.proto:272:3: "StepStats" is not defined.

In my WORKSPACE file, I have the following:

local_repository(
  name = "tf",
  path = __workspace_dir__ + "/serving/tensorflow",
)

local_repository(
  name = "tf_serving",
  path = __workspace_dir__ + "/serving",
)

load('//serving/tensorflow/tensorflow:workspace.bzl', 'tf_workspace')
tf_workspace("serving/tensorflow/", "@tf")

My hypothesis is that because tensorflow is a sub-sub-project it is not putting its generated files in the grandparent-projects bazel-out. However, I've tried many things and haven't been able to get it to work.

Answer 1

I have tensorflow serving in a folder and my project in another. This is my WORKSPACE file:

workspace(name = "my_project")

local_repository(
  name = "org_tensorflow",
  path = __workspace_dir__ + "/tf-serving/tensorflow/",
)

local_repository(
  name = "tf_serving",
  path = __workspace_dir__ + "/tf-serving/",
)

load('//tf-serving/tensorflow/tensorflow:workspace.bzl', 'tf_workspace')
tf_workspace("tf-serving/tensorflow/", "@org_tensorflow")

# ===== gRPC dependencies =====

bind(
    name = "libssl",
    actual = "@boringssl_git//:ssl",
)

bind(
    name = "zlib",
    actual = "@zlib_archive//:zlib",
)

I have also copied zlib.BUILD from tensorflow serving to the same place where I have the WORKSPACE file.

The BUILD file in my project has this rule (similar to yours):

py_binary(
    name = "export_model",
    srcs = [
        "export_model.py",
    ],
    deps = [
        "@org_tensorflow//tensorflow:tensorflow_py",
        "@tf_serving//tensorflow_serving/session_bundle:exporter",
    ],
)

The difference between my code and yours is that I included the dependencies in my root WORKSPACE. This code is compiling and working fine for me in one machine I have some issues to compile it in others (ubuntu 14.04) because of one dependency. I hope it works for you.