Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does the .NET Streams save memory?

Example:

  1. If I read a file and copy it to another file through .NET Streams will the total size of the file occupy the memory at any moment? Or will the bytes be discarded as soon as they are used?
  2. If the naive approach doesn't saves memory would buffered Streams do it?
like image 847
Jader Dias Avatar asked Feb 26 '09 19:02

Jader Dias


People also ask

Do streams store data?

A stream does not store data and, in that sense, is not a data structure.

What is stream memory?

Abstract. Memory streaming operations (i.e., memory-to-memory data transfer with or without simple arithmetic/logical operations) are one of the most important tasks in general embedded/mobile computer systems. In this paper, we propose a technique to accelerate memory streaming operations.

Does Java stream save memory?

No storage. Streams don't have storage for values; they carry values from a source (which could be a data structure, a generating function, an I/O channel, etc) through a pipeline of computational steps.

What is .NET stream?

Streams involve three fundamental operations: Reading - transferring data from a stream into a data structure, such as an array of bytes. Writing - transferring data to a stream from a data source. Seeking - querying and modifying the current position within a stream.


2 Answers

If you stream your copying, i.e. read a buffer, write a buffer, read a buffer, write a buffer etc until you've run out of data, it will only take as much memory as the buffer size. I expect File.Copy to do this (in the native Windows code, admittedly).

If you want to do it yourself, use something like this:

public void CopyData(Stream input, Stream output)
{
    byte[] buffer = new byte[32 * 1024];
    int read;
    while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
    {
        output.Write(buffer, 0, read);
    }
}

This will only take 32K however big the stream is.

EDIT: As noted in the comments, the streams may well have their own buffers as well, but the point is that you could still transfer a very large file without running out of memory.

like image 149
Jon Skeet Avatar answered Nov 15 '22 10:11

Jon Skeet


If you call something like ReadToEnd() then yes, the contents of the file will be loaded into memory. You're correct in guessing that using a buffer is the appropriate approach to ensure that only a subset of the file's data is loaded into memory at any given time.

like image 21
Ken Browning Avatar answered Nov 15 '22 10:11

Ken Browning