Binary data over websocket without encoding to UTF-8 or base64

Question

I am trying to use websockets in binary mode to transfer binary data from a server written in Python to a client browser (running javascript). I have implemented this communication in text mode, but now I wish to boost the performance by communicating in binary mode instead.

All the examples I have seen (eg this, and also if one dig through the source code of matplotlib/webagg which uses Tornado) claim they are using binary mode, but in fact they seem to encode the raw binary data into UTF-8 (or base64) at some point before transmission. In my view doing this is not true binary transfer as it adds 30% to 50% overhead.

So my question is, do one necessarily have to encode the binary data into utf-8 or base64 in order to use IP based websockets? If not, please point me to an example where this is done without encoding.

I have always believed that sockets do support true binary communication but maybe this is not the case for IP websockets for some reason. Perhaps someone can shed light onto the matter. There seem to have been advances in the past year or so in this area so that adds to the confusion.

Answer 1

The IETF 6455 WebSocket Protocol supports direct send/receive of binary data (the older Hixie protocol variant did not). If you are implementing your own framing, then you just need to set the opcode in the frame to 0x2 to indicate that the payload is raw binary data rather than UTF-8 encoded text. If you are using a python WebSocket library then you need to use the API provided by that library to select binary mode (if the library supports it).

Note: the example you linked to is not a WebSocket example (it is just a regular TCP client and server). Also, that example is not encoding the data as UTF-8 or base64. Websockify is a WebSocket server that supports direct binary data (in addition to base64 encoding for the older Hixie variant). Disclaimer: I created websockify.