SCons to generate variable number of targets

Question

I am trying to get SCons to generate multiple targets (number unknown directly in SConscript).

I have directory like:

headers/
  Header1.h
  Header2.h
  Header3.h
  Header4.h
meta/
  headers_list.txt

Now I want SConscript to read headers_list.txt, basing on its contents pick files from headers/ directory (i.e. it might contain only Header1 and Header3), for each of those I want to generate source using some function.

I have been trying to use env.Command to do that, but the issue is that it requires caller to specify targets list which for obvious reasons is not known when invoking env.Command.

The only thing I can think of is running:

for header in parse( headers_file ):
    source = mangle_source_name_for_header( header )
    env.Command( source, header, generator_action )

But this means I will be running parse( headers_file ) each time I invoke scons. If parsing is costly and the file is not often changed this step could be easily cached.

What SConsc construct/class/technique I am missing to achieve that caching?

edit:

It seems my question is similar to Build-time determination of SCons targets, but isn't there a technique without artificial dummy file?

Also, even with temporary file, I don't see how I am supposed to pass target variable from Command that generates variable number of targets to second one that would iterate over them.

edit 2:

This looks promising.

Answer 1

The only way I found I can do it is with emitter. Below example consists of 3 files:

./
|-SConstruct
|-src/
| |-SConscript
| |-source.txt
|-build/

SConstruct

env = Environment()

dirname = 'build'
VariantDir(dirname, 'src', duplicate=0)

Export('env')

SConscript(dirname+'/SConscript')

src/SConscript

Import('env')

def my_emitter( env, target, source ):
    data = str(source[0])
    target = []
    with open( data, 'r' ) as lines:
        for line in lines:
           line = line.strip()
           name, contents = line.split(' ', 1)
           if not name: continue

           generated_source  = env.Command( name, [], 'echo "{0}" > $TARGET'.format(contents) )
           source.extend( generated_source )
           target.append( name+'.c' )

    return target, source

def my_action( env, target, source ):
    for t,s in zip(target, source[1:]):
        with open(t.abspath, 'w') as tf:
            with open(s.abspath, 'r') as sf:
                tf.write( sf.read() )

SourcesGenerator = env.Builder( action = my_action, emitter = my_emitter )
generated_sources = SourcesGenerator( env, source = 'source.txt' )

lib = env.Library( 'functions', generated_sources )

src/source.txt

a int a(){}
b int b(){}
c int c(){}
d int d(){}
g int g(){}

Output:

$ scons
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
echo "int a(){}" > build/a
echo "int b(){}" > build/b
echo "int c(){}" > build/c
echo "int d(){}" > build/d
echo "int g(){}" > build/g
my_action(["build/a.c", "build/b.c", "build/c.c", "build/d.c", "build/g.c"], ["src/source.txt", "build/a", "build/b", "build/c", "build/d", "build/g"])
gcc -o build/a.o -c build/a.c
gcc -o build/b.o -c build/b.c
gcc -o build/c.o -c build/c.c
gcc -o build/d.o -c build/d.c
gcc -o build/g.o -c build/g.c
ar rc build/libfunctions.a build/a.o build/b.o build/c.o build/d.o build/g.o
ranlib build/libfunctions.a
scons: done building targets.

Also this has one thing I don't really like, which is parsing of headers_list.txt with each scons execution. I feel like there should be a way to parse it only if the file changed. I could cache it by hand, but I still hope there is some trick to make SCons handle that caching for me.

And I couldn't find a way to not duplicate files (a and a.c being the same). One way would be to simply generate library in my_action instead of sources (which is approach I used in my final solution).