protobuf over TCP vs grpc

Question

I need to stream a ton of data, and performance is the top priority. gRPC seems to be doing way more than just serializing protobuf, and honestly, it feels a bit bloated for what I need. Is it anti-pattern to manually serialize/deserialize the protobuf and send the bytes over TCP? And how much of a performance boost could I expect from doing that?

Answer 1

Manual protobuf serialization over TCP? Totally fine. People overcomplicate this stuff.

Basically, gRPC is like bringing a tank to a go-kart race. If you just need to move some bytes fast, just do that. Serialize your protobuf, send the bytes, done.

# Dead simple
sock.send(your_message.SerializeToString())

That's it. No rocket science. You'll probably get like 30% better performance by skipping all the gRPC overhead. HTTP/2, service discovery, all that jazz - great for big distributed systems, total overkill if you're just moving data between two points. Just make sure you handle your socket connection and maybe add a little length prefix so you know exactly how many bytes to read. But seriously, it's not complicated. Want me to show you a quick example of how to do it right?

Quick Code Example

import socket
from google.protobuf import your_message_pb2

def send_protobuf(sock, message):
    data = message.SerializeToString()
    sock.sendall(len(data).to_bytes(4, 'big') + data)

def receive_protobuf(sock, message_class):
    length = int.from_bytes(sock.recv(4), 'big')
    data = sock.recv(length)
    
    message = message_class()
    message.ParseFromString(data)
    return message