Python socket receiving multiple packets at once

Question

When I connect my Python client to the server, two packets are being sent to the client:

First one is:

FD 01

Second one:

FF 66 46 3E 61 37 07 CA 0B

However, when I'm trying to receive them in my Python client through sockets, I receive both at once:

FD 01 FF 66 46 3E 61 37 07 CA 0B

I want to receive packets into my buffer each after eachother, so I could parse one packet, do some job in the background and parse another packet in queue. How can I solve this?

This is my client code:

class ReceivePacket():
    def __init__(self, bytes):
        reply = str(bytes).encode('hex')
        print "<- [{}] - {}".format(headers.RECV.get(int(reply[:2], 16), int(reply[:2], 16)),
                                    ' '.join([reply[i:i + 2] for i in range(0, len(reply), 2)]).upper())

class Client(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

        self.size = 1024
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # type: socket.socket

        self.buf = bytearray(self.size)

        self.net = network.Network()
        self.net.bindClient(self)

        try:
            self.sock.connect((HOST, AUTH_PORT))
        except socket.error, msg:
            raise

    def run(self):
        while True:
            reply = self.sock.recv_into(memoryview(self.buf))

            if reply:
                self.receive(reply)

    def receive(self, nbytes):
        ReceivePacket(self.buf) # Having FD 01 FF 66 46 3E 61 37 07 CA 0B here

c = Client()
c.start()

Answer 1

What you can do is prepend all of your bytearrays with the length of the following bytearray. Just like in a tcp/udp packet you define your own header. For your bytestreams, one byte seems enough to hold the size of the message that comes after.

Your bytestreams will look something like this: 02 FD 01 09 FF 66 46 3E 61 37 07 CA 0B Resulting in this: 02 FD 01 09 FF 66 46 3E 61 37 07 CA 0B This allows you to receive all data in a buffer and then process the n bytes that come after.

If you also want the packet order to be able to be processed at random, you might want to also throw a second byte in the header to define the message type. The resulting bytestream would then look like this: [length|type|data]