Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Most efficient way of copying a raw byte array into an empty byte vector

Tags:

c++

stl

vector

I have a scenario in which I need to copy the contents of a raw dynamically allocated uint8_t array into a vector (which is guaranteed to be empty whenever this scenario happens).

vector<uint8_t> myVector;
const uint8_t* myRawArray;

It is really important to me that the copy operation is as efficient as possible and portable (various compiler versions might be used).

One approach I thought of using is this:

myVector.reserve(byteCount);
myVector.insert(myVector.begin(), myRawArray, myRawArray + byteCount);

Any ideas on how the speed of that compares to this one:

myVector.resize(byteCount);
memcpy(myVector.data(), myRawArray, byteCount);

I guess memcpy should be fast but then I am forced to use resize which needs to zero-out the memory, so I guess it will slow it down a bit..

Also, any other suggestions?

like image 265
mk33 Avatar asked Jan 01 '16 00:01

mk33


People also ask

How do you copy byte array?

copyOf(byte[] original, int newLength) method copies the specified array, truncating or padding with zeros (if necessary) so the copy has the specified length. For all indices that are valid in both the original array and the copy, the two arrays will contain identical values.

Is vector more efficient than array?

Vector occupies more memory. Array is memory efficient data structure. Vector takes more time in accessing elements. Array access elements in constant time irrespective of their location as elements are arranged in a contiguous memory allocation.

Which is faster vector or array?

A std::vector can never be faster than an array, as it has (a pointer to the first element of) an array as one of its data members. But the difference in run-time speed is slim and absent in any non-trivial program. One reason for this myth to persist, are examples that compare raw arrays with mis-used std::vectors.


2 Answers

If you don't need to create the vector before the copy takes place, you could always pass the raw array to the constructor of your vector:

std::vector<uint8_t> myVector(myRawArray, myRawArray + byteCount);

If you do need to construct the vector beforehand, the following is an option:

std::vector<uint8_t> myVector;
// ... do some stuff ...
// Now, we're ready for the copy, and byteCount is known.
myVector.reserve(byteCount);
std::copy(myRawArray, myRawArray + byteCount, std::back_inserter(myVector));

I would suggest using std::copy unless memcpy is proven to be faster. std::copy is safer and more idiomatic in C++ code, but don't be afraid to use memcpy if it really is proven to be faster. The speed difference will most likely change with different compilers.

I hope this helps.

like image 96
Ryan McCleary Avatar answered Sep 18 '22 15:09

Ryan McCleary


memcpy() is usually written in assembly and it is very optimized so you should know that memcpy will be fast. vector::insert is usually implemented as having a call to memcpy under the hood but it does need to check if there is enough space in the vector for the insertions to take place without any reallocations. I have not profiled this but I bet the first version with the call to reserve is faster.

An alternative to this would be to use std::copy which has been found to be slightly faster than using memcpy in some cases, you can be sure that if possible it also makes a call to memcpy or does something better. So performance issues should not be a problem with it. It will also take care of increasing the size of the vector to match your requirement.

like image 39
Curious Avatar answered Sep 21 '22 15:09

Curious