What are the differences between TPL Dataflow(TDF) and Reactive Extensions?

Question

After days of googleing I think I can't decide which one is for what scenario. Of course I would like to use a perfect framework which combines both (unrealistic of course). I even know that it's possible to use them together. But the real question is what are those core design elements in each one that make it impossible to emulate one with the other. These are the ones I found:

• in RX there is virtual time so the scheduler effectively controls the entire network but this is not possible in TDF because every block uses different tasks and they run independently
• in TDF a block can retry receiving a message/resend and stuff like that but in RX it's not possible.
• in rx the dataflow is serialized but in TDF this is optional

It would be nice to have some comprehension that is not list-like rather something that tries to derive both from a common abstract class/category talking only about the structural differences.

Answer 1

I think the best way to deal with this is to focus just on the first line of your question - deciding which one to use for a given scenario.

The two libraries serve different purposes. Speculating on core elements of the design of each to look for differences feels like asking why chalk doesn't taste like cheese - although I can't fault the specific points you have listed.

In my experience, there are rarely non-trivial scenarios where they are interchangeable in any sensible way. I think the actual descriptions from their documentation make a pretty self-explanatory answer to this question:

Rx

Reactive Extensions (Rx) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators. Using Rx, developers represent asychronous data streams using LINQ operators, and parameterize the concurrency in the asynchronous data streams using Schedulers. Simply put, Rx = Observables + LINQ + Schedulers

I would also look at my other answer here which is quite relevant when considering Rx: Where to draw the line with reactive programming. The essence of this is that Rx is good for responding to events you don't control in a timely manner.

TPL Dataflows

The Task Parallel Library (TPL) provides dataflow components to help increase the robustness of concurrency-enabled applications. These dataflow components are collectively referred to as the TPL Dataflow Library. This dataflow model promotes actor-based programming by providing in-process message passing for coarse-grained dataflow and pipelining tasks. The dataflow components build on the types and scheduling infrastructure of the TPL and integrate with the C#, Visual Basic, and F# language support for asynchronous programming. These dataflow components are useful when you have multiple operations that must communicate with one another asynchronously or when you want to process data as it becomes available. For example, consider an application that processes image data from a web camera. By using the dataflow model, the application can process image frames as they become available. If the application enhances image frames, for example, by performing light correction or red-eye reduction, you can create a pipeline of dataflow components. Each stage of the pipeline might use more coarse-grained parallelism functionality, such as the functionality that is provided by the TPL, to transform the image.