Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get contiguous Memory<byte> from ReadOnlySequence<byte>

After a wonderful introduction to pipes on Marc Gravell's blog post on the subject, I am tinkering with implementing pipes with sockets.

I know that Marc has already come up with Pipelines.Sockets.Unofficial , and I am using that as a reference, but I have a question.

It seems that SocketAsyncEventArgs has a new overload to the SetBuffer() method: SetBuffer(Memory<byte>)

It seems that the intent here is to integrate nicely with Pipes.

My confusion arises from the fact that Pipe.Reader.ReadAsync() returns a ReadResult containing a ReadOnlySequence<byte> (ReadResult.Buffer)

In the case that Buffer.IsSingleSegment == true, it's fairly obvious what to do:

SocketAsyncEventArgs.SetBuffer(Buffer[0])

But in the case where there are multiple segments, I'm not entirely sure what the best course of action is.

I could of course just get a byte[] from the pipe and be done with it, but that would incur a copy (possible more than one, even).

What is the intended use of ReadOnlySequence<byte> here? Is there a way to get `Memory' representing the whole contents of the sequence?

Perhaps I need to re-re-read Marc's blog post...

like image 554
Matthew Goulart Avatar asked Aug 10 '18 18:08

Matthew Goulart


1 Answers

For what I see: There is this ReadOnlySequence<>.First which in case of IsSingleSegment should give you ReadOnlyMemory<> you are looking for. For sequences with more segments I think you should iterate through them with

var iSegment = seq.Start;
ReadOnlyMemory<byte> readMemory;
while(seq.TryGet(ref iSegment, out readMemory, advance: true))
{ /* do smth */ }

Alternatively I think, this does the same thing:

foreach(var memory in seq)
{ /* do smth */ }

It makes sense to iterate through if you care about performance and don't want to make a copy of the whole buffer (that is what .ToArray() extension method does).
The intention IMHO was to effectively represent non-continuous buffers.
I am not sure about Sequence<> implementation details, but it is common in most of buffering scenarios to hold couple of small buffers(3, 2) and switch between them so to not allocate new memory unnecessarily. That is sometimes abstracted as "Circular" or "Ring" buffers.

It leads to such "sequential" memory access pattern as in Sequence<>.

like image 123
Arek Bal Avatar answered Nov 01 '22 10:11

Arek Bal