TPL Dataflow: Flatten incoming collection to sequential items

Question

I'm building an application using TPL dataflow. Actually I've the following problem. I've one transformblock var tfb1 = new TranformBlock<InMsg, IReadOnlyCollection<OutMsg>>. So tfb1 receives on in message and creates a list of out-messages. This list of out-messages shall be linked to a router datablock, which receives OutMsg as input (and not IReadOnlyCollection<OutMsg>).

How can I flatten the IReadOnlyCollection so that the containing message can be used as input for e.g. a transform block in the form of TransformBlock<OutMsg, SomeOtherType>. Is it possible via LinkTo()?

Thx

Answer 1

You can use the TransformManyBlock instead of TransformMany to flatten any IEnumerable<T> result, eg:

var flattenBlock = new TransformManyBlock<InMsg,OutMsg>(msg=>{
    List<OutMsg> myResultList;
    //Calculate some results
    return myResultList;
});

This will pass individual OutMsg instances to the next block.

You can use an iterator so individual messages are propagated immediatelly :

  var flattenBlock = new TransformManyBlock<InMsg,OutMsg>(msg=>{
    //Calculate some results
    foreach(var item in someDbResults)
    {
        yield return item;
    }
});

Or you can just flatten the input :

var flattenBlock = new TransformManyBlock<IEnumerable<OutMsg>,OutMsg>(items=>items);

You can link this block to any block that accepts OutMsg :

var block = new ActionBlock<OutMsg>(...);

flattenBlock.LinkTo(block);

You can route messages by passing a predicate to LinkTo. For example, if you want to route failure messages to a logging block, you can type:

flattenBlock.LinkTo(logBlock,msg=>msg.HasError);
flattenBlock.LinkTo(happyBlock);

Messages that don't match any predicate will get stuck in the output buffer and prevent the block from completing. By having one link without predicate you ensure that all messages will be processed