Real Hierarchical Builds with SCons?

Question

So I've read the questions on here about hierarchical builds like: Creating a Hierarchical Build with SCons

I want to do real hierarchical construction of two standalone repos that both use scons that I set up as sub-repos using mercurial. Below is the file layout that illustrates what I want to do.

Desired Layout:

project_root/  (new project that builds bar app using the libfoo built from source)

    libfoo_subrepo/  (standalone project repo from bitbucket)
        src/
            SConscript
            libfoo.c
            libfoo.h
        test/
            SConscript
            test_foo.c
        SConstruct

    barapp_subrepo/  (standalone project repo from bitbucket that uses libfoo)
        src/
            SConscript
            bar.c
            bar.h
        test/
            SConscript
            test_bar.c
        SConstruct

    test/
        SConscript
        test_bar_with_foo.c
    SConstruct

So I have two separate repos, both using scons. The first one, libfoo, can be cloned standalone and built using scons. When running scons in the libfoo root directory, it builds a static library of libfoo in src/ and builds a unit test executable in test/ that links against the static library in src/.

The second repo has the bar app that depends on libfoo. It too can be cloned standalone and if libfoo is installed on the build system, it can be built using scons.

What I want to do, is set up a new repo (project_root) that has both the libfoo and bar app repos set as subrepos using mercurial. So when you clone this new repo, it automatically pulls down bar app and it's dependency, libfoo. Then I want to be able to run scons in the root of this new repo and have it execute scons in the libfoo_subrepo/ root to build libfoo and it's unit tests. Then I want it to run scons in the barapp_subrepo/ root to build bar and tell it to link against the libfoo static library in libfoo_subrepo/src/. Lastly, I want it to build some new unit tests in tests/ that use both the libfoo static library and the source files from bar app to test bar app and libfoo when combined together.

As far as I can tell from reading the scons documentation is that I would need to create a custom Builder for "subrepo" that would run scons in a sub-shell. Then I could add libfoo.subrepo and barapp.subrepo to the project_root/ directory and some how rig it so that when the builder goes to execute the command to build libfoo.subrepo that it translates the source name into a path that it executes scons in.

building 'libfoo.subrepo' translates into executing 'cd libfoo_subrepo; scons'

It looks to me like scons cannot recursively build standalone scons projects. Everything I've read assumes you have the ability to create SConscript files in the subfolders and then have the root SConstruct file depend on the SConscript files. Please tell me there is a way to do what I want with scons. I don't want to go back to make.

Thanks.

Answer 1

Im not sure why you would need to make a custom builder, if I understand you correctly, I think everything you need can be done with SCons and its builtin builders.

To do what you explain, you would indeed need 3 Seperate SConsctruct files, to be able to do 3 seperate builds. I would also add 3 SConscript files and make all of them as follows:

Edit: In this example, its better to create the Environment() in the SConstruct scripts

project_root/SConstruct

# This SConstruct orchestrates building 3 subdirs

import os

subdirs = ['libfoo_subrepo', 'barapp_subrepo', 'test']
env = Environment()

for subdir in subdirs:
    SConscript(os.path.join(subdir, 'SConscript'), exports = ['env'])

libfoo_subrepo/SConstruct

# This SConstruct does nothing more than load the SConscript in this dir
# The Environment() is created in the SConstruct script
# This dir can be built standalone by executing scons here, or together
# by executing scons in the parent directory
env = Environment()
SConscript('SConscript', exports = ['env'])

libfoo_subrepo/SConscript

# This SConstruct orchestrates building 2 subdirs
import os

Import('env')
subdirs = ['src', 'test']

for subdir in subdirs:
    SConscript(os.path.join(subdir, 'SConscript'), exports = ['env'])

barapp_subrepo/SConstruct

# This SConstruct does nothing more than load the SConscript in this dir
# The Environment() is created in the SConstruct script
# This dir can be build standalone by executing scons here, or together
# by executing scons in the parent directory
env = Environment()
SConscript('SConscript', exports = ['env'])

barapp_subrepo/SConscript

# This SConstruct orchestrates building 2 subdirs
import os

Import('env')
subdirs = ['src', 'test']

for subdir in subdirs:
    SConscript(os.path.join(subdir, 'SConscript'), exports = ['env'])

I hope the comments in each file explains its purpose.

Hope this helps.