how to deal with the position in a c# stream

Question

The (entire) documentation for the position property on a stream says:

• When overridden in a derived class, gets or sets the position within the current stream.
• The Position property does not keep track of the number of bytes from the stream that have been consumed, skipped, or both.

That's it. OK, so we're fairly clear on what it doesn't tell us, but I'd really like to know what it in fact does stand for. What is 'the position' for? Why would we want to alter or read it? If we change it - what happens?

In a pratical example, I have a a stream that periodically gets written to, and I have a thread that attempts to read from it (ideally ASAP). From reading many SO issues, I reset the position field to zero to start my reading. Once this is done:

• Does this affect where the writer to this stream is going to attempt to put the data? Do I need to keep track of the last write position myself? (ie if I set the position to zero to read, does the writer begin to overwrite everything from the first byte?)
• If so, do I need a semaphore/lock around this 'position' field (subclassing, perhaps?) due to my two threads accessing it?
• If I don't handle this property, does the writer just overflow the buffer?

Perhaps I don't understand the Stream itself - I'm regarding it as a FIFO pipe: shove data in at one end, and suck it out at the other. If it's not like this, then do I have to keep copying the data past my last read (ie from position 0x84 on) back to the start of my buffer?

I've seriously tried to research all of this for quite some time - but I'm new to .NET. Perhaps the Streams have a long, proud (undocumented) history that everyone else implicitly understands. But for a newcomer, it's like reading the manual to your car, and finding out:

The accelerator pedal affects the volume of fuel and air sent to the fuel injectors. It does not affect the volume of the entertainment system, or the air pressure in any of the tires, if fitted.

Technically true, but seriously, what we want to know is that if we mash it to the floor you go faster..

EDIT - Bigger Picture

I have data coming in either from a serial port, a socket, or a file, and have a thread that sits there waiting for new data, and writing it to one or more streams - all identical.
One of these streams I can access from a telnet session from another pc, and that all works fine.
The problem I'm having now is parsing the data in code in the same program (on another of the duplicated streams). I'm duplicating the data to a MemoryStream, and have a thread to sit and decipher the data, and pass it back up to the UI. This thread does a dataStream.BeginRead() into it's own buffer, which returns some(?) amount of data up to but not more than the count argument. After I've dealt with whatever I got back from the BeginRead, I copy the remaining data (from the end of my read point to the end of the stream) to the start of my buffer so it won't overflow.
At this point, since both the writing and reading are asynchronous, I don't know if I can change the position (since it's a 'cursor' - thanks Jon). Even if send a message to the other thread to say that I've just read 28 bytes, or whatever - it won't know which 28 bytes they were, and won't know how to reset it's cursor/position. I haven't subclassed any streams - I've just created a MemoryStream, and passed that to the thread that duplicates the data out to whatever streams are needed.

This all feels too complex to be the right way of doing it - I'm just unable to find a simple example I can modify as needed..

How else do people deal with a long-term sporadic data stream that needs to be send to some other task that isn't instantaneous to perform?

---

EDIT: Probable Solution

While trying to write a Stream wrapper around a queue due to information in the answers, I stumbled upon this post by Stephen Toub.
He has written a BlockingStream, and explains:

Most streams in the .NET Framework are not thread safe, meaning that multiple threads can't safely access an instance of the stream concurrently and most streams maintain a single position at which the next read or write will occur. BlockingStream, on the other hand, is thread safe, and, in a sense, it implicitly maintains two positions, though neither is exposed as a numerical value to the user of the type. BlockingStream works by maintaining an internal queue of data buffers written to it. When data is written to the stream, the buffer written is enqueued. When data is read from the stream, a buffer is dequeued in a first-in-first-out (FIFO) order, and the data in it is handed back to the caller. In that sense, there is a position in the stream at which the next write will occur and a position at which the next read will occur.

This seems exactly what I was looking for - so thanks for the answerrs guys, I only found this from your answers.

Answer 1

I think that you are expecting a little too much from the documentation. It does tell you exactly what everything does, but it doesn't tell you much about how to use it. If you are not familiar with streams, reading only the documention will not give you enough information to actually understand how to use them.

Let's look at what the documentation says:

"When overridden in a derived class, gets or sets the position within the current stream."

This is "standard documentation speak" for saying that the property is intended for keeping track of the position in the stream, but that the Stream class itself doesn't provide the actual implementation of that. The implementation lies in classes that derive from the Stream class, like a FileStream or a MemoryStream. Each have their own system of maintaining the position, because they work against completely different back ends.

There can even be implementation of streams where the Position property doesn't make sense. You can use the CanSeek property to find out if a stream implementation supports a position.

"The Position property does not keep track of the number of bytes from the stream that have been consumed, skipped, or both."

This means that the Position property represents an absolute position in the back end implementation, it's not just a counter of what's been read or written. The methods for reading and writing the stream uses the position to keep track of where to read or write, it's not the other way around.

For a stream implementation that doesn't support a position, it could still have returned how many bytes have been read or written, but it doesn't. The Position property should reflect an actual place in the data, and if it can't do that it should throw a NotSupportedException exception.

Now, let's look at your case:

Using a StreamReader and a StreamWriter against the same stream is tricky, and mostly pointless. The stream only has one position, and that will be used for both reading and writing, so you would have to keep track of two separate positions. Also, you would have to flush the buffer after each read and write operation, so that there is nothing left in the buffers and the Position of the stream is up to date when you retrieve it. This means that the StreamReader and StreamWriter can't be used as intended, and only act as a wrapper around the stream.

If you are using the StreamReader and StreamWriter from different threads, you have to synchronise every operation. Two threads can never use the stream at the same time, so a read/write operation would have to do:

• lock
• set position of the stream from local copy
• read/write
• flush buffer
• get position of the stream to local copy
• end lock

A stream can be used as a FIFO buffer that way, but there are other ways that may be better suited for your needs. A Queue<T> for example works as an in-memory FIFO buffer.

Answer 2

• The position is the "cursor" for both writing and reading. So yes, after resetting the Position property to 0, it will start overwriting existing data
• You should be careful when dealing with a stream from multiple threads in the first place, to be honest. It's not clear whether you've written a new Stream subclass, or whether you're just the client of an existing stream, but either way you need to be careful.
• It's not clear what you mean by "If I don't handle this property" - what do you mean by "handle" here? Again, it would help if you were clearer on what you were doing.

A Stream may act like a pipe... it really depends on what you're doing with it. It's unclear what you mean by "do I have to keep copying the data past my last read" - and unclear what you mean by your buffer, too.

If you could give an idea of the bigger picture of what you're trying to achieve, that would really help.