Apache Thrift maximum message size

Question

We are using Apache Thrift to exchange messages between two systems. In one of the message we are exchanging a list (c++) which can become huge in size. Can you please let me know what is the maximum message size we can exchange using Apache Thrift?

Answer 1

There is no defined "per se" limit (at least none that I am aware of). It mostly depends on how the data are held in memory, what load is on the server and how much resources are avilable. For the most part, contiguos blocks of memory (RAM) will very likely become the scarcest resource, so we should focus on that point.

The "how the data are held in memory" refers to the fact that for the sake of better throughput some transports (buffered, framed) tend to allocate more memory and larger blocks than others. Depending on the language's implementation this process may be implemented more or less efficient in terms of memory cost.

If you really plan to transfer large blocks of data, you should also look at other options like

• chunking the data into blocks
• sending/returning only an URL or LAN share through the service, instead of the whole data