Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Way to know if TPL Dataflow Block busy?

TPL Dataflow block has .InputCount and .OutputCount properties. But it can perform execution over item right now, and there is no property like .Busy [Boolean]. So is there a way to know if block is now operating and one of item still there?

enter image description here

UPDATE:

Let me explain my issue. Here on pic is my current Dataflow network scheme. BufferBlock holds URLs to load, number of TransformBlocks load pages through proxy servers and ActionBlock at the end performs work with loaded pages. TransformBlocks has predefined .BoundedCapacity, so BufferBlock waits for any of TransformBlocks becomes free and then post item into it.

Initially I post all URLs to Buffer Block. Also if one of TransformBlocks throw exception during loading HTML, it returns it's URL back to BufferBlock. So my goal is somehow wait until all of my URLs was guarantee loaded and parsed. For now I'm waiting like this:

Do While _BufferBlock.Count > 0 Or _ 
         GetLoadBlocksTotalInputOutputCount(_TransformBlocks) > 0 Or _ 
         _ActionBlock.InputCount > 0

        Await Task.Delay(1000)
Loop

Then I call TransformBlock.Complete on all of them. But in this case, there still can be last URLs loading it TransformBlocks. If last URL was not successfully loaded, it becomes 'lost', because none of TransformBlocks wouldn't take it back. That's why I want to know if TransformBlocks are still operating. Sorry my bad English.

enter image description here

like image 266
AsValeO Avatar asked May 01 '14 11:05

AsValeO


1 Answers

Even if you could find out whether a block is processing an item, it wouldn't really help you achieve your goal. That's because you would need to check the state of all the blocks at exactly the same moment, and there is no way to do that.

What I think you need is to somehow manually track how many items have been fully processed and compare that with the total number of items to process.

You should know the number of items to process from the start (it's you who sends them to the buffer block). To track the number of items that have been fully processed, you can add a counter to your parsing action block (don't forget to make the counter thread-safe, since your action block is parallel).

Then, if the counter reaches the total number of items to process, you know that all work is done.

like image 114
svick Avatar answered Nov 14 '22 20:11

svick