How does Rx behave when stream of data comes faster than Subscribers can consume?

Question

I am very excited about using Rx in production application; where I will be listening to incoming notification updates coming from different channel.

I will be writing Rx query on top this stream where I will throttling using .Window() operator. Subscriber (In my case it is ActionBlock) will process this data in blocking fashion; (i.e it will not spawn Task from ActionBlock). Keeping above in mind if data comes at much faster rate than what my Subscriber can consume then what will happen to incoming data. Does Rx query uses any buffer internally; will it get overflowed ?

Answer 1

The phenomenon you're referring to is called Back Pressure, and the Rx team is currently exploring different ways to handle this situation. One solution might be communicating back-pressure back to the Observable so that it might "slow down".

To alleviate back-pressure, you could use lossy operators such as Throttle or Sample.

Timothy's answer is mostly right, but it is possible to have back-pressure occur on a single thread. This can happen if you use asynchronous code. In that sense, back-pressure is related to synchronization and scheduling, not threading (recall that by default Rx is single threaded).

If you run into a scenario where events are being produced faster than they can be consumed, and you're not using a lossy operator to alleviate the back-pressure, those items are usually being scheduled/queued/buffered, which can lead to a lot of memory allocation.

Personally, this has not been an issue for me, since usually events are processed faster than they are yielded, or loss of events is simply not an option, and therefore the extra memory consumption is inevitable.