Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to treat a std::vector like a C buffer efficiently?

Tags:

c++

c++11

stl

With C buffer, I often do like this:

BYTE buffer[MAX_SIZE];
int dataSize = 0;

while (appRunning()) 
{
    dataSize += Receive(buffer + dataSize, MAX_SIZE - dataSize);

    int processedSize = ProcessBuffer(buffer, dataSize);
    ASSERT(processedSize <= dataSize);
    dataSize -= processedSize;
    memmove(buffer, buffer + processedSize, dataSize);
};

Is it possible to do so with a std::vector without losing much performance?

EDIT: I have found a way to replace a raw C buffer by a std::vector.

std::vector<BYTE> vbuf;
vbuf.reserve(MAX_SIZE); // allocated at once

while (appRunning()) 
{
    int pendingSize = GetPendingDataSize(); // from a socket
    if (pendingSize > vbuf.capacity())
        pendingSize = vbuf.capacity();

    vbuf.resize(pendingSize);
    int recvSize = Receive(vbuf.data(), vbuf.size());
    ASSERT(recvSize < vbuf.size());
    int processedSize = ProcessBuffer(vbuf.data(), vbuf.size());
    std::rotate(vbuf.begin(), vbuf.begin() + processedSize, vbuf.end());
    vbuf.resize(vbuf.size() - processedSize);
};

Actually, in my practical usage, receiving data and processing data may be done in multithread. So by using vector, I do not need to manage buffer's allocation, data size and buffer capacity manually. Compares to the C buffer, the performance penalty here is at vbuf.resize() calls. But I think that penalty is insignificant. Any better way is appreciated.

like image 464
SteveH Avatar asked Jun 19 '15 09:06

SteveH


People also ask

Is Vector faster than set C++?

Vector is faster for insertion and deletion of elements at the end of the container. Set is faster for insertion and deletion of elements at the middle of the container.

Is std :: array faster than vector?

A std::vector can never be faster than an array, as it has (a pointer to the first element of) an array as one of its data members. But the difference in run-time speed is slim and absent in any non-trivial program. One reason for this myth to persist, are examples that compare raw arrays with mis-used std::vectors.

Is vector Push_back slow?

In short, push_back is doing more than what operator[] is doing - which is why it is slower (and more accurate).


1 Answers

When receiving messages over a TCP connection the last message in the buffer may be incomplete. After processing complete messages, people often just memmove that last incomplete message to the beginning of the buffer.

Another strategy is to use a "smart" ring-buffer to avoid that memmove and also avoid data wrapping over the ring buffer thus creating discontinuity. To make a "smart" ring-buffer allocate memory for the buffer using mmap and map the same region of pages twice with no gaps in between. This way reading past the end of the buffer continues reading it from the start, preventing the discontinuity inherent when using a regular ring-buffer.


Using std::vector for network buffers is less than ideal because resizing a vector initialises its elements that are later overwritten by recv call. That initialization is unnecessary for such buffers.

like image 172
Maxim Egorushkin Avatar answered Sep 30 '22 15:09

Maxim Egorushkin