Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Send files through RabbitMQ

Is it a good idea to send files with size about 1Mb through RabbitMQ? I want to send message in json format with binary fields corresponding to the files.

And how to do it properly using spring-amqp? Just by publishing object with next class?

class Message {
    String field1;
    byte[] fileField1;
    byte[] fileField2;
}
like image 770
Nawa Avatar asked Nov 27 '14 16:11

Nawa


People also ask

How do you send pictures on RabbitMQ?

In general, if you want to send a file, you have to read it and send the buffers. You should send the file using more publish and not just one, then recreate the file on the consumer side. You should avoid sending big buffer.

Is RabbitMQ push or pull?

RabbitMQ uses a push-based model with a smart producer, which means the producer decides when to push data. A prefetch limit is defined on the consumer to stop the producer from overwhelming consumers.


1 Answers

I would suggest not only reading those links that were posted but also, doing some of your own experimentation. The thing I would be concerned about is performance at the service level and at the client level.

You might want to consider having a server host the files/data and allow rabbitmq just send the message to the consumer with the id of the message in it. So when your consumer gets the message, it sends an HTTP GET request to a service that requests the actual message payload. That way RabbitMQ stays lightweight. You can always add consumers and servers if you need.

That's my opinion without experimenting. You might find that it's still lighting fast with 1MB payloads. That's why I would say to experiment and find out for yourself.

Hope you find this helpful!

like image 144
Josh Chappelle Avatar answered Sep 20 '22 01:09

Josh Chappelle