Flask Docker container SocketIO Issues

Question

I have a Flask application that uses SocketIO to fetch data from Postgres live.

The app works fine when I run this locally.

The problem arouses when I use docker-compose to host my Flask app. My JS client and flask server is hosted into a single app and on the same container.

My socketio in JS is like this:

var socket = io().connect(window.location.protocol + '//' + document.domain + ':' + location.port);

Dockerfile:

# Using python 3.7 in Alpine
FROM python:3.6.5-stretch

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
ADD . /app

RUN apt-get update -y && apt-get upgrade -y

# Install the dependencies from requirements
RUN pip install -r requirements.txt

# Tell the port number the container should expose
EXPOSE 8083

# Run the command
ENTRYPOINT ["./entry.sh"]

entry.sh:

#!/bin/sh
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -b :8083 -w 5 run:app

My docker-compose is as such:

version: "3.8"

services:
  fortweet:
    container_name: fortweet
    build: ./
    env_file:
      - secret.env
    networks:
      - plutusnet
    ports:
      - 8083:8083
    restart: always

networks:
  plutusnet:
    name: plutus_network
    driver: bridge

I've also tried to use var socket = io.connect('http://public_ip_of_website:8083') but my socket connection still doesn't work.

How it should normally work is when i run this locally and click a certain button, it executes this function in my JS:

$("#tweets-live-start").click(function(){
    if (is_streaming == true){
      alert("A stream is already running")
    }else{
      $.ajax({
        type: "POST",
        url : "/admin/startstream",
        data: {url : "print \"hello\""},
        contentType: 'application/json;charset=UTF-8'
      });
    }
  });

When my server gets the hello, it starts a tweet streaming and emits them through the socket. Then my socket captures them as such:

// Listens for tweets
socket.on('stream-results', function(results){

  // Insert tweets in divs
  $('#live-tweet-container').prepend(`
  <div class="row justify-content-md-center mt-3">
    <div class="col-md-2">
        <img width="56px" height="56px"  src="${results.profile_pic !== "" ? results.profile_pic : "/static/icons/profile-pic.png"}" class="mx-auto d-block rounded"  alt="">
    </div>
    <div class="col-md-8 my-auto">
      <div><b>${results.author}</b></div>
      <div>${results.message}</div>
    </div>
  </div>
  `);
});

But when i run it on docker, nothing happens.

When i check my browser JS console, it seems that it is polling with a bad request and i don't know why:

index.js:83 GET http://th3pl4gu3.com:8083/socket.io/?EIO=3&transport=polling&t=NPYYrxr 400 (BAD REQUEST)

Here is my docker ps for more info:

46446efeb472   mervin16/fortweet:dev   "/bin/bash entry.sh"  About a minute ago   Up About a minute   0.0.0.0:8083->8083/tcp   fortweet

12b2bff36af0   postgres    "docker-entrypoint.s…"   2 hours ago          Up 2 hours          0.0.0.0:5432->5432/tcp   plutus

I don't think this is an accessibility issue because i tried several telnet tests from each container and to each container also.

I checked logs of the docker container and it gives this:

fortweet    | [2020-12-26 15:18:55 +0000] [8] [INFO] Starting gunicorn 20.0.4
fortweet    | [2020-12-26 15:18:55 +0000] [8] [INFO] Listening at: http://0.0.0.0:8083 (8)
fortweet    | [2020-12-26 15:18:55 +0000] [8] [INFO] Using worker: geventwebsocket.gunicorn.workers.GeventWebSocketWorker
fortweet    | [2020-12-26 15:18:55 +0000] [11] [INFO] Booting worker with pid: 11
fortweet    | [2020-12-26 15:18:55 +0000] [12] [INFO] Booting worker with pid: 12
fortweet    | [2020-12-26 15:18:55 +0000] [13] [INFO] Booting worker with pid: 13
fortweet    | [2020-12-26 15:18:55 +0000] [14] [INFO] Booting worker with pid: 14
fortweet    | [2020-12-26 15:18:55 +0000] [15] [INFO] Booting worker with pid: 15
fortweet    | The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)
fortweet    | The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)
fortweet    | 172.16.0.1 - - [2020-12-26 15:19:57] "POST /admin/startstream HTTP/1.1" 204 170 0.023672
fortweet    | The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)
fortweet    | The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:20] "GET /socket.io/?EIO=3&transport=polling&t=1608996021267-7 HTTP/1.1" 400 195 0.001418
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:20] "GET /socket.io/?EIO=3&transport=polling&t=1608996021267-7 HTTP/1.1" 400 195 0.001418
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:21] "GET /socket.io/?EIO=3&transport=polling&t=1608996021395-8 HTTP/1.1" 400 195 0.001625
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:21] "GET /socket.io/?EIO=3&transport=polling&t=1608996021395-8 HTTP/1.1" 400 195 0.001625
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:26] "GET /socket.io/?EIO=3&transport=polling&t=1608996026417-9 HTTP/1.1" 400 195 0.001367
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:26] "GET /socket.io/?EIO=3&transport=polling&t=1608996026417-9 HTTP/1.1" 400 195 0.001367
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:26] "GET /socket.io/?EIO=3&transport=polling&t=1608996027270-8 HTTP/1.1" 400 195 0.003811
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:26] "GET /socket.io/?EIO=3&transport=polling&t=1608996027270-8 HTTP/1.1" 400 195 0.003811
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:34] "POST /admin/startstream HTTP/1.1" 204 170 0.015831
fortweet    | 172.16.0.1 - - [2020-12-26 15:20:36] "GET /socket.io/?EIO=3&transport=polling&t=1608996036486-11 HTTP/1.1" 400 195 0.001096

FYI, the plutus container is just my Postgres database that my web app connects to.

Can anyone please help me ?

Answer 1

TL;DR - you're using incompatible socketIO versions between client and server. Check the table below and make sure you're using the appropriate python & javascript versions.

Like the container logs say, you're using incompatible versions of SocketIO between client and server. The SocketIO & EngineIO protocols have been through several revisions, and they're not all backward compatible, so you have to make sure you're using appropriate implementations of the protocol on the client and server side that can talk to one another.

I suspect that the reason it's working when you run your app locally but not in your Docker container is because the dependencies in your container's requirements.txt are referencing older, incompatible versions of the python implementation. Based on what you described, it seems like the locally installed python implementation of socketIO is a newer version, and therefore compatible with the newer JS version on client-side, and has no trouble connecting (or it could be vice-versa....older client, newer server).

If you're using the native javascript Socket.IO version 3.0+ on the client side (which implements the SocketIO Protocol v5 and EngineIO Protocol v4), then you need to make sure you're using the appropriate versions of the Python implementations on the server side. You didn't specify, but I assume you're using Flask-SocketIO, which itself is a wrapper around python-socketio, the actual Python implementation of the SocketIO protocol.

Check what version of SocketIO client you are using in your Javascript. Then check your requirements.txt and make sure the python-socketio version is compatible, per the below table (source):

JS SocketIO Version	SocketIO Protocol	EngineIO Protocol	python-socketio
0.9.x	1, 2	1, 2	Not supported
1.x and 2.x	3, 4	3	4.x
3.x	5	4	5.x

You're most likely using the JS version 3.x with the 4.x version on the python side (which are NOT compatible). Make sure that you are using Flask-SocketIO v5.x, python-socketio v5.x, and python-engineio v4.x, and that your JS client is 3.x. That should fix your problem.

If this is working properly in your local environment, then you could simply run pip freeze > requirements.txt and use that for your docker build. This requirements.txt file would have the correct dependencies since clearly it is working when you run locally.

Alternatively, if you want to ensure that you have the latest version of everything, you could run pip install --upgrade flask-socketio. This will install the latest version of Flask-SocketIO and the most up to date dependencies (including python-socketio and python-engineio). Then just regenerate your requirements.txt file and use it in your docker build.