Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Best way to read a large file into a byte array in C#?

I have a web server which will read large binary files (several megabytes) into byte arrays. The server could be reading several files at the same time (different page requests), so I am looking for the most optimized way for doing this without taxing the CPU too much. Is the code below good enough?

public byte[] FileToByteArray(string fileName) {     byte[] buff = null;     FileStream fs = new FileStream(fileName,                                     FileMode.Open,                                     FileAccess.Read);     BinaryReader br = new BinaryReader(fs);     long numBytes = new FileInfo(fileName).Length;     buff = br.ReadBytes((int) numBytes);     return buff; } 
like image 251
Tony_Henrich Avatar asked Jan 08 '10 21:01

Tony_Henrich


People also ask

What is the max size of byte array?

Max Size It generally depends on the JVM that we're using and the platform. Since the index of the array is int, the approximate index value can be 2^31 – 1. Based on this approximation, we can say that the array can theoretically hold 2,147,483,647 elements.

What is a Bytearray?

The bytearray type is a mutable sequence of integers in the range between 0 and 255. It allows you to work directly with binary data. It can be used to work with low-level data such as that inside of images or arriving directly from the network. Bytearray type inherits methods from both list and str types.

What is byte array in C?

byte array in CAn unsigned char can contain a value from 0 to 255, which is the value of a byte. In this example, we are declaring 3 arrays – arr1, arr2, and arr3, arr1 is initialising with decimal elements, arr2 is initialising with octal numbers and arr3 is initialising with hexadecimal numbers.


2 Answers

Simply replace the whole thing with:

return File.ReadAllBytes(fileName); 

However, if you are concerned about the memory consumption, you should not read the whole file into memory all at once at all. You should do that in chunks.

like image 151
mmx Avatar answered Sep 19 '22 22:09

mmx


I might argue that the answer here generally is "don't". Unless you absolutely need all the data at once, consider using a Stream-based API (or some variant of reader / iterator). That is especially important when you have multiple parallel operations (as suggested by the question) to minimise system load and maximise throughput.

For example, if you are streaming data to a caller:

Stream dest = ... using(Stream source = File.OpenRead(path)) {     byte[] buffer = new byte[2048];     int bytesRead;     while((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) {         dest.Write(buffer, 0, bytesRead);     } } 
like image 33
Marc Gravell Avatar answered Sep 17 '22 22:09

Marc Gravell