Understanding bjam's targets, and how to specify new ones?

Question

I'm having problems understanding how to specify and invoke targets with bjam. By this, I mean that I want to provide bjam with command-line targets to build (from a Makefile actually) that correspond to different aspects of the build process, rather than just running the whole thing.

For example, right now when I type 'bjam' it goes off and builds a python extension, runs a unit-test file, and also creates a separate 'main' executable. I have custom rules that do each step, and my Jamfile simply lists them in order:

project-name = example ;

sources =
  $(project-name).cpp
  $(project-name)_ext.cpp
  ;

build-ext $(project-name) : $(sources) ;

build-main $(project-name) ;

In my Jamroot (up one directory) I have these rules defined, here's the incomplete file:

# A rule to simplify declaration of extension tests:
rule run-test ( test-name : sources + )
{
    import testing ;
    testing.make-test run-pyd : $(sources) : : $(test-name) ;
}

# A rule to further simply declaration of extension tests:
rule run-ext-test ( project-name )
{
  run-test $(project-name) : $(project-name)_ext test_$(project-name)_ext.py ;
}

# A rule to simplify copying of the extension and Boost.Python libraries to the current directory
rule convenient-copy ( project-name )
{
  install convenient_copy
    : $(project-name)_ext
    : <install-dependencies>on <install-type>SHARED_LIB <install-type>PYTHON_EXTENSION
      <location>.
    ;
}

rule build-ext ( project-name : sources + )
{
  python-extension $(project-name)_ext : $(sources) : ;

  # copy the extension and Boost.Python libraries to the current directory
  convenient-copy $(project-name) ;

  # run extension tests
  run-ext-test $(project-name) ;
}

rule build-main ( project-name : other-sources * )
{
  obj $(project-name).o : $(project-name).cpp ;
  exe main_$(project-name) : main_$(project-name).cpp $(project-name).o $(other-sources) ;
  install main : main_$(project-name) : <location>. ;
}

However I've noticed that the following invocations of bjam don't do what I'd like them to do:

$ bjam build-main
notice: could not find main target build-main
notice: assuming it is a name of file to create.
don't know how to make <e>build-main
...found 1 target...
...can't find 1 target...

$ bjam main_example
...patience...
...patience...
...found 1597 targets...
...updating 3 targets...
gcc.compile.c++ bin/gcc-4.6/debug/main_example.o
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.link bin/gcc-4.6/debug/main_example
...updated 3 targets...

^^^ but the install rule wasn't run so the binary isn't copied to the Jamfile directory.

Oddly, there are some targets that do things, but not always what I expect:

$ bjam main
...patience...
...patience...
...found 1598 targets...
...updating 3 targets...
gcc.compile.c++ bin/gcc-4.6/debug/main_example.o
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.link main_example
...updated 3 targets...

That did create the binary, in the Jamfile directory.

Where did the main target come from? I didn't define it...

Another odd one:

$ bjam example_ext
...patience...
...patience...
...found 2834 targets...
...updating 3 targets...
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.compile.c++ bin/gcc-4.6/debug/example_ext.o
gcc.link.dll bin/gcc-4.6/debug/example_ext.so
...updated 3 targets...

^^^ created example_ext.so, but didn't copy it to the Jamfile location.

$ bjam example_ext.so
notice: could not find main target example_ext.so
notice: assuming it is a name of file to create.
...patience...
...patience...
...found 2836 targets...
...updating 4 targets...
gcc.compile.c++ bin/gcc-4.6/debug/example.o
gcc.compile.c++ bin/gcc-4.6/debug/example_ext.o
gcc.link.dll bin/gcc-4.6/debug/example_ext.so
common.copy example_ext.so
...updated 4 targets...

^^^ created the .so file and copied it, but didn't invoke convenient-copy to bring in the libboost_python.so files.

I really don't understand what's going on here. The bjam documentation is really causing me serious problems. It describes targets in detail, but in the context of rules, not in the context of invoking bjam from the command line. I did come across some mention of pseudotargets and 'generate' but it seemed far too complicated for what I feel should be a simple use-case. There was also mention of a 'bind' mechanism but the documentation mentions =$(BINDRULE[1])= which makes no sense to me.

I also came across aliases, NOTFILE and explicit but I wasn't sure if I was on the right track, and wasn't able to do anything conclusive.

Are there any good examples of how to create custom targets in bjam? Or am I simply trying to use bjam in ways that weren't intended?

Answer 1

A bjam invocation with no arguments builds everything because you don't have any target marked as explicit, which can be used to prevent some targets building unless explicitly requested.

bjam build-main fails as build-main is not a name of a target or file to generate; it is name of a rule (function) that can be called with different arguments, each call declaring different sets of targets.

bjam main_example makes the target that was declared with:

exe main_$(project-name) : main_$(project-name).cpp $(project-name).o $(other-sources) ;

which naturally doesn't incorporate the install target for it, declared on the next line.

bjam main builds main_example and installs it, because main is the name of its install target, declared with: install main : main_$(project-name) : <location>. ;

Note that had you called build-main more than once in your jamfile, every bjam invocation would exit with error: No best alternative for ./main, so it's better to rename the name of the install target to something like install_main_$(project-name) to prevent name collisions. Then bjam install_main_example will build and install main_example.

bjam example_ext makes the target that was declared with:

python-extension $(project-name)_ext : $(sources) : ;

and no install again like with bjam main_example.

bjam example_ext.so works as example_ext.so is indeed a file name that is being created (under the given platform of course), so all targets that result in files named example_ext.so will be generated by that invocation. Now this is why not all files that convenient-copy instructed to be installed were installed by bjam example_ext.so invocation. Here I want to clarify something: "didn't invoke convenient-copy" isn't quite an accurate terminology. convenient-copy is a name of a rule, not a target, and with the code as written above, that rule will always get called regardless of bjam invocation arguments. All it does when called is declaring a target named convenient_copy (note the underscore), which in turn results in (implicitly) declaring a few file targets (for install) e.g. example_ext.so, libboost_python.so and other dependencies that are matched by <install-type>SHARED_LIB <install-type>PYTHON_EXTENSION. When convenient-copy rule gets called, it doesn't actually build anything, it just declares some targets. What gets actually built is decided at a later phase, and the decision depends on bjam invocation arguments.

bjam convenient_copy would build example_ext.so and install it properly together with its dependencies, yet it suffers from the same problem that main install target does: it will break if you call convenient-copy more than once. Changing the name from convenient_copy to install_$(project-name)_ext would solve the issue, and then you would invoke the install with bjam install_example_ext.

And finally if you want a target to not build on bjam invocation with no arguments, you mark that target as explicit, e.g.

explicit install_main_$(project-name) ;

would suppress main_example's installation unless explicitly requested. To prevent main_example's build too, add explicit for main_$(project-name) and $(project-name).o as well:

explicit main_$(project-name) ;
explicit $(project-name).o ;

or all together:

explicit install_main_$(project-name) main_$(project-name) $(project-name).o ;

Note that declaring main_$(project-name) as explicit while not doing so for install_main_$(project-name), or declaring $(project-name).o as explicit while not doing so for main_$(project-name) doesn't make sense, as if a target is requested to build, all its dependency targets will be requested to build as well, even when those dependencies are explicit.