TPL Dataflow block has .InputCount
and .OutputCount
properties. But it can perform execution over item right now, and there is no property like .Busy [Boolean]
. So is there a way to know if block is now operating and one of item still there?
UPDATE:
Let me explain my issue. Here on pic is my current Dataflow network scheme.
BufferBlock
holds URLs to load, number of TransformBlock
s load pages through proxy servers and ActionBlock
at the end performs work with loaded pages. TransformBlock
s has predefined .BoundedCapacity
, so BufferBlock
waits for any of TransformBlocks
becomes free and then post item into it.
Initially I post all URLs to Buffer Block
. Also if one of TransformBlock
s throw exception during loading HTML, it returns it's URL back to BufferBlock
. So my goal is somehow wait until all of my URLs was guarantee loaded and parsed. For now I'm waiting like this:
Do While _BufferBlock.Count > 0 Or _
GetLoadBlocksTotalInputOutputCount(_TransformBlocks) > 0 Or _
_ActionBlock.InputCount > 0
Await Task.Delay(1000)
Loop
Then I call TransformBlock.Complete
on all of them. But in this case, there still can be last URLs loading it TransformBlock
s. If last URL was not successfully loaded, it becomes 'lost', because none of TransformBlocks wouldn't take it back. That's why I want to know if TransformBlock
s are still operating. Sorry my bad English.
Even if you could find out whether a block is processing an item, it wouldn't really help you achieve your goal. That's because you would need to check the state of all the blocks at exactly the same moment, and there is no way to do that.
What I think you need is to somehow manually track how many items have been fully processed and compare that with the total number of items to process.
You should know the number of items to process from the start (it's you who sends them to the buffer block). To track the number of items that have been fully processed, you can add a counter to your parsing action block (don't forget to make the counter thread-safe, since your action block is parallel).
Then, if the counter reaches the total number of items to process, you know that all work is done.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With