Dynamic task generators

Question

I am evaluating waf build for an existing project that has tasks similar to that:

1. preprocessing phase: datafile => TransformationTask => library name list
2. for each library name: 
  2.1 import files from repository
  2.2 build library

The library list depends on the preprocessing task and is naturally not known in advance.

How can this be achieved with waf?

Answer 1

You have to generate a file with the library list with a first task. Another task will have the output of the first as an input and will process the corresponding file if needed to generate what you need.

It is somehow the example given in §11.4.2 of the waf book. You have to replace the compiler output parsing with your library description file parsing. You need to copy the example and change the run method in mytool.py like:

class src2c(Task.Task):
    color = 'PINK'
    quiet = True
    before = ['cstlib']

    def run(self):

        libnode = self.inputs[0]
        libinfo = libnode.read_json()

        name = libinfo['name']
        files = [f"repo/{file}" for file in libinfo['files']]
        taskgen = self.generator
        
        # library name
        taskgen.link_task.outputs = []
        taskgen.link_task.add_target(name)
        # library sources files
        nodes = [taskgen.path.make_node(f) for f in files]
        # update discovered dependancies
        taskgen.bld.raw_deps[self.uid()] = [self.signature()] + nodes

        with g_lock:
            self.add_c_tasks(nodes)

    # cf waf book § 11.4.2
    def add_c_tasks(self, lst):
        ...
        
    # cf waf book § 11.4.2
    def runnable_status(self):
        ...

In the wscript, I simulate the datafile transformation with a copy.:

def options(opt):
    opt.load("compiler_c")

def configure(cnf):
    cnf.load("compiler_c")
    cnf.load("mytool", tooldir=".")

def build(bld):
    bld(source = "libs.json", target = "libs.src", features = "subst")
    bld(source = "libs.src", features = ["c", "cstlib"])

With a simple my_lib.json:

{
    "name": "mylib2",
    "files": ["f1.c", "f2.c"]
}

And files repo/f1.c and repo/f2.c like void f1(){} and void f2(){}