Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

End-to-End Reactive Streaming RESTful service

I have read this post and I don't know if it is a stupid question. I am completely newbie in reactive programming.

The question is very simple: supposing I have a fully-reactive back-end, how can I stream to browser (a large text, for instance), and print each chunk to the user as soon as they come from server?

Probably I am missing some important conceptual points, but all I need to know is: Can I send small parts of data (from server to browser), with one single HTTP GET (? or not) request? The key is: can I write those small parts, until the whole data is sent?

Thanks for any help. I really tried google before, but I always get information about other concepts (like websockets, long-polling, react.js), that I think it is not the case.

EDIT: I'am not asking for a specific API ou library. I just want to understand the concept, for example: "You CANNOT do this with HTTP protocol, period!" or "This has NOTHING to do with Reactive Programming, you are confused about what stream really means. See 'something_else.js' ".

EDIT2: I did a small rest controller using spring-mvc (spring-boot) with this method:

@RequestMapping(value = "/flush", method = RequestMethod.GET)
public void flushTest(HttpServletResponse response) {

    response.setContentType("text/html");
    response.setStatus(SC_OK);

    try (ServletOutputStream outputStream = response.getOutputStream()) {
        for (int i = 0; i < 3; i++) {
            String chunk = "chunk_" + i;
            outputStream.print(chunk);
            outputStream.flush();
            response.flushBuffer();
        }
    } catch (IOException e) {
        e.printStackTrace();
    }

}

In the browser, only the whole response arrived, even using the flushBuffer. After some research, that this could be a TOMCAT issue, I changed my pom.xml including undertow:

<dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-undertow</artifactId>
</dependency>

With default configurations:

server.undertow.accesslog.enabled=true
server.undertow.accesslog.dir=target/logs
server.undertow.accesslog.pattern=combined
server.compression.enabled=true
server.compression.min-response-size=1l

At this moment, using debug in my IDE, each chunk was being written after the flushBuffer.

So, I believe this could be a TOMCAT configuration. Long story short, I know there are a lot of things to help me on "streaming solutions", and thanks for all the comments, but my problem was a little bit more conceptual.

like image 525
Igor Veloso Avatar asked Feb 04 '23 15:02

Igor Veloso


1 Answers

There is a protocol on top of HTTP which allows delivery of a server response in smaller, ready-to-be-consumed, chunks to the browser using HTTP transport. It's called SSE (Server Sent Events) or EventSource, here is pretty thoroughly assembled article on the topic.

There are other means to stream data using the HTTP protocol. One such alternative is JSON streaming, in which you write partial responses on the wire from the server and consume the JSON chunks when they arrive. On the consuming side a popular library is called Oboe.js, on the server side you basically need to write the partial data on the response wire when you want to sent it, or when it's available.

For both approaches Rx is useful for handling the streaming logic on the server side. You model the pieces as a stream (handle errors, etc.) and eventually, in the Subscriber, you write the single emissions on the wire, closing the response when the Observable is finished.

On the client side, the EventSource events can be wrapped in a new Observable (via Rx.Observable.create()) and processed further as a stream. Also Oboe.js events can be converted into Observables (via Rx.Observable.fromEvent().

like image 182
mkulke Avatar answered Feb 07 '23 18:02

mkulke