Read data from a UDP socket with an unknown length

Question

How could I socket.ReadFromUDP without knowing the length beforehand? Right now I have something like this.

buff := make([]byte, <unknownLength>)
for {
    socket.ReadFromUDP(buff)
    fmt.Println("Bytes read:", buff.Len())
}

I tried this, but it always read 1024 bytes.

buff := make([]byte, 1024)
for {
    bytesRead, _, _ := socket.ReadFromUDP(buff)
    fmt.Println("Bytes read:", bytesRead)
}

The packets I'm receiving from the server vary from a couple of bytes to hundres of bytes.

Answer 1

A UDP socket receives messages in discrete datagrams (hence the name SOCK_DGRAM). Though Go has a relatively clean networking API, you still can't get away without knowing a little bit about the underlying sockets (How large should my recv buffer be when calling recv in the socket library).

A UDP socket will Read up to the size of the receiving buffer and will discard the rest, so you can't continue to read like you would on a TCP stream. A single receive on a UDP socket consumes a single datagram from the network.

Though the theoretical max payload of a UDP packet is 65,507, in practice you are probably not going to get messages over 1400 bytes because of path MTU limits. If you set your buffer considerably larger than this, to something like 4096 bytes you will be very safe, but you could use a 64k buffer to be certain.

Though it's not feasible in Go to try and determine the pending packet size, you can tell if you lost data for what it's worth:

oob := make([]byte, 1024)
n, on, flags, addr, err := c.ReadMsgUDP(buff, oob)
if flags & syscall.MSG_TRUNC != 0 {
    fmt.Println("truncated read")
}

Answer 2

The basic technique is to use a buffer one larger than the largest expected datagram. Then if you get a datagram that size you know it doesn't fit your application protocol and should be discarded, probably after logging it in case it's a programming error at the sender, or your application protocol is incompletely specified.