akka stream consume web socket

Question

Getting started with akka-streams I want to build a simple example. In chrome using a web socket plugin I simply can connect to a stream like this one https://blockchain.info/api/api_websocket via wss://ws.blockchain.info/inv and sending 2 commands

• {"op":"ping"}
• {"op":"unconfirmed_sub"} will stream the results in chromes web socket plugin window.

I tried to implement the same functionality in akka streams but am facing some problems:

• 2 commands are executed, but I actually do not get the streaming output
• the same command is executed twice (the ping command)

When following the tutorial of http://doc.akka.io/docs/akka/2.4.7/scala/http/client-side/websocket-support.html or http://doc.akka.io/docs/akka-http/10.0.0/scala/http/client-side/websocket-support.html#half-closed-client-websockets Here is my adaption below:

object SingleWebSocketRequest extends App {

implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()

import system.dispatcher

// print each incoming strict text message
val printSink: Sink[Message, Future[Done]] =
    Sink.foreach {
      case message: TextMessage.Strict =>
        println(message.text)
    }

      val commandMessages = Seq(TextMessage("{\"op\":\"ping\"}"), TextMessage("{\"op\":\"unconfirmed_sub\"}"))
      val helloSource: Source[Message, NotUsed] = Source(commandMessages.to[scala.collection.immutable.Seq])

      // the Future[Done] is the materialized value of Sink.foreach
      // and it is completed when the stream completes
      val flow: Flow[Message, Message, Future[Done]] =
      Flow.fromSinkAndSourceMat(printSink, helloSource)(Keep.left)

      // upgradeResponse is a Future[WebSocketUpgradeResponse] that
      // completes or fails when the connection succeeds or fails
      // and closed is a Future[Done] representing the stream completion from above
      val (upgradeResponse, closed) =
      Http().singleWebSocketRequest(WebSocketRequest("wss://ws.blockchain.info/inv"), flow)

      val connected = upgradeResponse.map { upgrade =>
        // just like a regular http request we can access response status which is available via upgrade.response.status
        // status code 101 (Switching Protocols) indicates that server support WebSockets
        if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
          Done
        } else {
          throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
        }
      }

      // in a real application you would not side effect here
      // and handle errors more carefully
      connected.onComplete(println) // TODO why do I not get the same output as in chrome?
      closed.foreach(_ => println("closed"))
    }

when using the flow version from http://doc.akka.io/docs/akka-http/10.0.0/scala/http/client-side/websocket-support.html#websocketclientflow modified as outlined below, again, the result is twice the same output:

{"op":"pong"}
{"op":"pong"}

See the code:

object WebSocketClientFlow extends App {
  implicit val system = ActorSystem()
  implicit val materializer = ActorMaterializer()
  import system.dispatcher

  // Future[Done] is the materialized value of Sink.foreach,
  // emitted when the stream completes
  val incoming: Sink[Message, Future[Done]] =
    Sink.foreach[Message] {
      case message: TextMessage.Strict =>
        println(message.text)
    }

  // send this as a message over the WebSocket
  val commandMessages = Seq(TextMessage("{\"op\":\"ping\"}"), TextMessage("{\"op\":\"unconfirmed_sub\"}"))
  val outgoing: Source[Message, NotUsed] = Source(commandMessages.to[scala.collection.immutable.Seq])
  //  val outgoing = Source.single(TextMessage("hello world!"))

  // flow to use (note: not re-usable!)
  val webSocketFlow = Http().webSocketClientFlow(WebSocketRequest("wss://ws.blockchain.info/inv"))

  // the materialized value is a tuple with
  // upgradeResponse is a Future[WebSocketUpgradeResponse] that
  // completes or fails when the connection succeeds or fails
  // and closed is a Future[Done] with the stream completion from the incoming sink
  val (upgradeResponse, closed) =
    outgoing
      .viaMat(webSocketFlow)(Keep.right) // keep the materialized Future[WebSocketUpgradeResponse]
      .toMat(incoming)(Keep.both) // also keep the Future[Done]
      .run()

  // just like a regular http request we can access response status which is available via upgrade.response.status
  // status code 101 (Switching Protocols) indicates that server support WebSockets
  val connected = upgradeResponse.flatMap { upgrade =>
    if (upgrade.response.status == StatusCodes.SwitchingProtocols) {
      Future.successful(Done)
    } else {
      throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
    }
  }

  // in a real application you would not side effect here
  connected.onComplete(println)
  closed.foreach(_ => {
    println("closed")
    system.terminate
  })
}

How can I achieve the same result as in chrome

• display print of subscribed stream
• at best periodically send update (ping statements) as outlined in https://blockchain.info/api/api via {"op":"ping"}messages

Note, I am using akka in version 2.4.17 and akka-http in version 10.0.5

Answer 1

A couple of things I notice are:

1) you need to consume all types of incoming messages, not only the TextMessage.Strict kind. The blockchain stream is definitely a Streamed message, as it contains loads of text and it will be delivered in chunks over the network. A more complete incoming Sink could be:

  val incoming: Sink[Message, Future[Done]] =
    Flow[Message].mapAsync(4) {
      case message: TextMessage.Strict =>
        println(message.text)
        Future.successful(Done)
      case message: TextMessage.Streamed =>
        message.textStream.runForeach(println)
      case message: BinaryMessage =>
        message.dataStream.runWith(Sink.ignore)
    }.toMat(Sink.last)(Keep.right)

2) your source of 2 elements might complete too early, i.e. before the websocket responses come back. You can concatenate a Source.maybe by doing

val outgoing: Source[Strict, Promise[Option[Nothing]]] =
    Source(commandMessages.to[scala.collection.immutable.Seq]).concatMat(Source.maybe)(Keep.right)

and then

  val ((completionPromise, upgradeResponse), closed) =
    outgoing
      .viaMat(webSocketFlow)(Keep.both)
      .toMat(incoming)(Keep.both)
      .run()

by keeping the materialized promise non-complete, you keep the source open and avoid the flow shutdown.