Extending Flask REST API with WebSockets

Question

I am currently working on extending my existing REST API created using Flask-RESTPlus with WebSocket support. The idea is to create a Web Thing Model compliant Web Thing (Gateway). The "Things" in my use-case are dynamically added or removed.

The current setup allows a consumer to fetch the latest values from a Thing, e.g. temperature sensor, using a HTTP GET request to /thingId/properties/temperature. The values are in reality consumed from Kafka and temporarily stored in Redis.

Now I am wondering how I can extend this setup and allow the consumer to not only poll the latest values, but subscribe to the property of a Thing using WebSockets. I have a working solution where I create "Rooms" for each property, but this requires two separate servers and duplication of endpoints.

For REST I have

@app.route('/<thingId>/properties/<propertyId>')
    # get latest datapoint
    return latestDatapoint

For Flask-SocketIO I have

@socketio.on('join')
def on_join(data):
    username = data['username']
    room = data['room'] # e.g. /thingId/properties/temperature
    join_room(room)
    send(username + ' has entered the room.', room=room)

and then I'm forwarding the data to the correct room as it comes in from Kafka. On the client side I then need to connect to the WebSocket server and join the room

socket.on('connection', function(socket){
  socket.emit('join', 'some room');
});

This implementation works, but I was strongly hoping for an alternative workflow as shown in the figure below where the client connects to the same endpoint used in the REST API, but with the WebSocket protocol instead of joining rooms etc.

Do you have any idea if this already exists or is feasible to implement?

Answer 1

I have a working solution where I create "Rooms" for each property, but this requires two separate servers and duplication of endpoints.

The Socket.IO server and your HTTP server do not necessarily need to be separate, In all supported configurations you can host HTTP and Socket.IO applications with a single server.

I also don't see the endpoint duplication, but maybe this is because you think of Socket.IO event handlers as endpoints, while in fact they are not. With Socket.IO there is a single endpoint in the HTTP sense, since all Socket.IO traffic travels on a single URL. Your event handlers are just that, functions that are invoked when certain events pop up on the Socket.IO endpoint.

where the client connects to the same endpoint used in the REST API, but with the WebSocket protocol instead of joining rooms etc.

So you want your client to establish a separate WebSocket connection for each thing it wants to watch? That seems a bit resource intensive and not very scalable to me. If the client needs to watch 100 things, then it will have to maintain 100 WebSocket connections. Keep in mind that most browsers cap the number of WebSocket connections they can have open at a time, both per page and globally.

Socket.IO is a higher-level protocol that is built on top of WebSocket and HTTP. If you still prefer to use WebSocket directly, then you can take any of the available open source WebSocket servers and implement your application with that instead of Socket.IO. Here are a few options for Python off the top of my mind:

• eventlet
• gevent-websocket
• websockets (asyncio)
• Tornado (asyncio)

You are going to lose a few things that Socket.IO offers that are very handy:

• Automatic reconnections
• Automatic support for non-WebSocket clients via long-polling
• Event-based dispatching
• Rooms

So you need to make sure these are not important features, or you are okay implementing them yourself directly on the WebSocket server.