How can I publish or subscribe to a materialized Akka Stream flow graph?

Question

I'm playing around with Akka Stream and I'm trying to figure out its flexibility after materialization.

One way to do so is to use the low level reactive streams API: http://doc.akka.io/api/akka-stream-and-http-experimental/1.0-M3/#akka.stream.scaladsl.PublisherSource

However, you need to define these points to publish or subscribe to. Is there a way to publish or subscribe to an arbitrary materialized flow graph node? This should be possible, since a materialized flow graph is nothing more than a collection of actors.

For example: First, deploy flow graph 1: A ~> B ~> C

Then, deploy flow graph 2 and 3: D ~> B B ~> E

Answer 1

It is not possible to completely dynamically change structures of stream processing pipelines. We do foresee certain kinds of dynamic processing stages (like an "fanout to workers which may come and go"), but in general streams and stream processing pipelines should have a defined layout before materialisation. This also makes sense because of the back-pressure mechanisms employed in akka-streams and reactive-streams in general – it has to be carefully managed internally, and allowing arbitrary interactions is just not something this streaming model is designed for.

If you want arbitrary interactions, Actors should suit you better. If you want to have a processing element that can take external signals to steer the processing pipeline, it would be a special element inside that materialised pipeline designed to take in these signals, not just any element.

---

Since I published this post we added some dynamic features, which may be of interest to people who land on this site, most notably the MergeHub and the BroadcastHub

Please note that as ow writing the stable version of Akka Streams is 2.4.16.