Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

FileStream Read/Write method's limitation

FileStream's read/write method can take only integer value as length. But FileStreamobject returns length in long. In this case, what if file size is larger than integer value (approximate more than 2GB). Then how FileStream's read/write method handle long value.

like image 288
jams Avatar asked Apr 13 '11 19:04

jams


People also ask

What is the difference between MemoryStream and FileStream?

As the name suggests, a FileStream reads and writes to a file whereas a MemoryStream reads and writes to the memory. So it relates to where the stream is stored.

What is buffer size in FileStream?

The size of the buffer used by FileStream for buffering. The default buffer size is 4096.

What is the use of FileStream?

Use the FileStream class to read from, write to, open, and close files on a file system, and to manipulate other file-related operating system handles, including pipes, standard input, and standard output.

How does FileStream work C#?

The FileStream is a class used for reading and writing files in C#. It is part of the System.IO namespace. To manipulate files using FileStream, you need to create an object of FileStream class. This object has four parameters; the Name of the File, FileMode, FileAccess, and FileShare.


2 Answers

Then you read and write in multiple chunks. The CLR has a limit on the size of any particular object anyway (also around 2GB IIRC, even on a 64-bit CLR), so you couldn't have a byte array big enough for it to be a problem.

You should always loop when reading anyway, as you can't guarantee that a Read call will read as many bytes as you requested, even if there's more data to come.

EDIT: Reading in chunks:

byte[] buffer = new byte[1024 * 32];
int bytesRead;

while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0)
{
    // Use the data you've read
}

Writing in chunks will depend on what you're writing... it's hard to talk about it in the abstract.

like image 57
Jon Skeet Avatar answered Oct 26 '22 22:10

Jon Skeet


There is no need to directly write more than 2Gb of data in one call

If you would really have that amount buffered contiguously in memory (? maby as an unsafely acquired UnmanagedMemoryStream to implement a core dump?) you could easily batch the writes in multiple calls. It will get written to disk in blocks of 512k to max 4k on current hardware, anyway.

The great value of 'streaming' interfaces is that you can have it anywhich way. In fact, when you look into to it you will find that the CLR Arrays (and anything else) are actually bounded to 2GB

Update

Since you have now confessed that you basically want to copy streams, you might be better served with an instant solution. There is File.Copy

File.Copy("file-a.txt", "file-new.txt");

Or there is the standard answer

Stream input
input.CopyTo(output); // .NET 4.0

// .NET 3.5 and others
public static void CopyStream(Stream input, Stream output)
{
    byte[] buffer = new byte[32768];
    while (true)
    {
        int read = input.Read (buffer, 0, buffer.Length);
        if (read <= 0)
            return;
        output.Write (buffer, 0, read);
    }
}

Don't forget about Flushing, Closing and Disposing your Streams as appropriate if you are handling the streams manually.

Cheers

like image 29
sehe Avatar answered Oct 27 '22 00:10

sehe