Python: Socket and threads?

Question

I have a design problem:

I have two threads, a heartbeat/control thread and a messagehandler thread.

Both are sharing the same socket, however the messageHandler thread only sends out messages and never receives. The heartbeat thread sends and receives (receives messages and reacts on heartbeats).

The problem is I'm not sure if this is safe. There is no mechanism, I myself, implemented to see if the socket is being used. So is sharing a socket over python automatically thread safe or not?

Also if it's not, the reason I put them in a separate thread is because the heartbeat is more important than the message handling. This means that if it gets flooded with messages, it still needs to do a heartbeat. So if I have to implement a bolt, is there away I can prioritize if my heartbeat/control thread needs to send a heartbeat?

Answer 1

A better way instead of using a third thread is to use threading.Lock() to protect the socket resource because this removes the need of a third thread. You have lower overhead and less latency than if you had a third thread.

import threading

lock = threading.Lock()

def sendfunction(sock, data):
    with lock:
        sock.send(data)

You can call that from either of your threads but only one thread at a time will be allowed to call sock.send. When a thread reaches the lock that is already locked by another thread it will sleep until the other thread releases the lock then it will acquire the lock and the process repeats.

The threading module contains Lock, RLock, and Condition which are all very useful when dealing with multiple threads and you will find it well worth your time to become familiar with them and their usage.

You could incorporate the heartbeat into your message handling by checking the current time with the last time you sent a heartbeat before you process each message, and that would prevent being flooded with messages causing a heartbeat not to be sent. The problem is if your message handling code does not run then no heartbeats will be sent. You could alleviate this by having your message handling code get a dummy message on interval to allow it to check if it needs to send a heartbeat and just ignore the dummy message.

You should try to use threads sparingly (aim for a single thread) however in your case a thread would likely be okay since it is going to spend most of it's time sleeping. You should however not use a daemon thread because they do not properly shutdown. Although the damage might be nonexistent in your case if it did not properly shutdown it still might throw some type of fault (error message) which looks bad.

I do not agree with the multiple sockets method as I think it would actually complicate the situation. You will find a many types of network services/applications out there that incorporate the heartbeat and messages into a single socket byte stream.

Answer 2

Unfortunately,The socket shared by multi-thread is not thread safe.Think about buffer two threads operate on with no lock.

The normal way to implement is with two socket,just like what ftp does.cmd socket and msg socket.

If you wanna implement this by one socket,you can put different type msgs into different queues,and with a third thread consumes the queue and send them through the only socket.

In this way,you can control heartbeat msg priory to data msg.