Passing progress messages from server to frontend in Rails 4

Question

I'm working on an Rails app which features a custom paperclip processor which accepts video files and manipulates them into various other formats.

As I'm working with video and this process takes a while to complete on the server, I would like to pass information about the status of the processing from the server to the frontend as it's happening.

My question is: what is the best method for getting these messages from my custom paperclip class to my JS without page reload? I'm guessing some kind of event pub/sub system but am unclear on the best method of implementing this.

Sorry if this is a bit of a noob question, been away from Rails for a while!

Answer 1

Your question was relatively vague, so apologies if my answer is equally:

---

WebSockets Or SSE (Server Sent Events)

We just implemented something similar to this with an app which deletes multiple records, and sends updates on which have been deleted

You're right with pub/sub, and I'll attempt to explain the underlying tech. It's actually a similar process for SSE or WebSockets

---

Receiving "Live" Data

The process of receiving "live" data from the server is to use the ActionController::Live controller inside Rails. This works with the response.headers['Content-Type'] = 'text/event-stream' HTTP mime-type to send small packets of data to a Javascript "listener" (which you initialize on your front-end)

Whether you use SSE or WebSockets, you have to have a "live" controller action (which creates the live-information), and also have a Javascript EventListener in place to capture the real-time data in the browser

The way you do this changes with each technology, but it's actually a similar process for both SSE & WebSockets

---

SSE's

Server Sent Events is an HTML5 technology which basically allows you to receive (not send) updated data by opening an asynchronous connection to an endpoint (your Rails ActionController::Live action) on your server, which will publish the data you need

You'd typically set up this using Redis (to compile the data into JSON) and send it using Rails' ActionController::Live functionality. Your Javascript will "listen" for the event being triggered, thus allowing you to capture the real-time data sent from the server & manipulate it on the front end

The problem with SSE's is that it uses long-polling to continually "ping" the endpoint in an attempt to get the latest information. Although Puma allows for multi-threading, we were unable to get it to run the connection concurrently, degrading performance significantly (basically, sending a request every second eats up your connections)

---

WebSockets

Websockets are a similar technology to SSE's, but with a major difference -- they allow a perpetual connection:

WebSocket is a protocol providing full-duplex communications channels over a single TCP connection

This means that you can send and receive data, as well as only connecting once & maintaining the connectivity, regardless of what you do interim. We found this to be a far more stable solution than SSE's, as it removes the requirement for long-polling, and, if you can keep authentication correct, gives you the ability to receive data correctly

All the big-hitters use WebSockets (including StackOverflow), and it's actually relatively simple to set up:

---

"Real Time" Setup

You need:

1. Rails ActionController::Live Function & Endpoint (route)
2. Third-Party Websocket Integration (Pusher)
3. Javascript EventListeners (to handle returned data)

Here's some live code we use:

#app/controllers/resources_controller.rb (inherited resources)
        def destroy        
            element = @resource.find(params[:id])
            element.destroy
            Pusher['private-user-' + current_user.id.to_s].trigger('my_event', {
                message: "Removed #{element.title}"
            })
        end

   #app/assets/javascripts/users.js.coffee.erb
   pusher = new Pusher("******************",
        cluster: 'eu'
   )

   channel = pusher.subscribe("private-user-#{gon.user_id}")
   channel.bind "my_event", (data) ->
       alert data.message

---

This opens a persisting connection to the pusher app, which then handles our data. Whenever pusher receives our data, it sends the update to us, allowing us to alert the user to the message

I know it's a simplistic overview, but I hope it helps