When to use byte array, and when to use stream?

Question

I need to send images and small video files (around 5MB, less than 10MB) to a REST service, which I will write. I am wondering whether I should use Byte[] or Stream to accomplish this task. What would be the dividing line in terms of transfer file size between using Byte[] and Stream?

Answer 1

The general test when choosing between a byte array or stream depends on whether you know, up front, how many bytes there are in the data, and if this number is a reasonably small one for the given purpose.

For example, if you're dealing with a small icon file (less than 50KB) available on the local machine and you know the filesize, go with a byte array.

Conversely, if you're working with a movie file where it would be difficult, not to mention unnecessary, to keep 2GB of content in memory at once, use a stream.

Streams are best for handling larger sets of data or data whose length is not known up front.

Answer 2

Ultimately, you're going to be sending a stream either way.

If you received the data from another source outside of your control as the massive byte[], then you might as well keep it in that form for your processing unless it's inconvenient to you, and let it be pushed into the network stream down the line.

If you receive it as a stream, there's no point turning it into a massive byte[] just to have it pushed to another stream. Use a buffer of 4 or 8 kiB (4 or 8 rather than 4 to 8, as there are some minor advantages in the whole-number-of-memory-page coincidences of those that you may as well take advantage of).

If you're creating it yourself, the stream is both easier for most cases (wrap in a binary or text writer, and work through that) as well as more efficient.

More generally, if I see a buffer of more than 8kiB that is being written to or read from a stream, then I'll note that as the first thing to try changing if things seem to be too slow.

Answer 3

The amount of free memory you're willing to commit to the transaction is your only real constraint.

If you have a 5M file, then you'll need to load the entire thing in to RAM, which will cost 5M.

If you stream it, you can use far less memory, by reading small chunks from the file in to a reusable buffer and writing those chunks to the HTTP stream.