Is it possible to initialize std::vector over already allocated memory?

Question

My question is fairly simple and I am quite surprised I can't find anything related. Probably it is easy or totally stupid (or I can't search).

As the title says, is it possible to use std::vector on already allocated memory, so it does not allocate new elements from start but uses what is given. I would imagine it as something like:

T1 *buffer = new T1[some_size]; std::vector<T2> v(buffer, some_size); // <- ofc doesn't work 

The opposite is quite simple and (maybe not pretty but) works:

std::vector<T2> v(some_size); T1 *buffer = &v[0]; 

The storage is guaranteed to be continuous, so it is as safe as an iterator.

My motivation is quite simple. I pass around some raw memory data, i.e. bytes, and since I know their interpretation at some other places I would like to convert them back to something meaningful. I could do a reinterpret_cast and use normal c-style array, but I prefer c++ facilities.

I get the feeling this should be safe given that we give up ownership of buffer to vector, because it needs to be able to reallocate.

Answer 1

Like this.. Containers in the standard usually take an allocator. Using c++11's allocator traits, it is very easy to create an allocator as you don't have to have all the members in the allocator. However if using an older version of C++, you will need to implement each member and do the rebinding as well!

For Pre-C++11, you can use the following:

#include <iterator> #include <vector> #include <iostream>  template<typename T> class PreAllocator {     private:         T* memory_ptr;         std::size_t memory_size;      public:         typedef std::size_t size_type;         typedef ptrdiff_t difference_type;         typedef T* pointer;         typedef const T* const_pointer;         typedef T& reference;         typedef const T& const_reference;         typedef T value_type;           PreAllocator(T* memory_ptr, std::size_t memory_size) throw() : memory_ptr(memory_ptr), memory_size(memory_size) {};         PreAllocator (const PreAllocator& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};          template<typename U>         PreAllocator (const PreAllocator<U>& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};          template<typename U>         PreAllocator& operator = (const PreAllocator<U>& other) {return *this;}         PreAllocator<T>& operator = (const PreAllocator& other) {return *this;}         ~PreAllocator() {}          pointer address (reference value) const {return &value;}         const_pointer address (const_reference value) const {return &value;}          pointer allocate (size_type n, const void* hint = 0) {return memory_ptr;}         void deallocate (T* ptr, size_type n) {}          void construct (pointer ptr, const T& val) {new (ptr) T (val);}          template<typename U>         void destroy (U* ptr) {ptr->~U();}         void destroy (pointer ptr) {ptr->~T();}          size_type max_size() const {return memory_size;}          template<typename U>         struct rebind         {             typedef PreAllocator<U> other;         }; };  int main() {     int my_arr[100] = {0};     std::vector<int, PreAllocator<int> > my_vec(PreAllocator<int>(&my_arr[0], 100));     my_vec.push_back(1024);     std::cout<<"My_Vec[0]: "<<my_vec[0]<<"\n";     std::cout<<"My_Arr[0]: "<<my_arr[0]<<"\n";      int* my_heap_ptr = new int[100]();     std::vector<int, PreAllocator<int> > my_heap_vec(PreAllocator<int>(&my_heap_ptr[0], 100));     my_heap_vec.push_back(1024);     std::cout<<"My_Heap_Vec[0]: "<<my_heap_vec[0]<<"\n";     std::cout<<"My_Heap_Ptr[0]: "<<my_heap_ptr[0]<<"\n";      delete[] my_heap_ptr;     my_heap_ptr = NULL; } 

For C++11, you can use the following:

#include <cstdint> #include <iterator> #include <vector> #include <iostream>  template <typename T> class PreAllocator {     private:         T* memory_ptr;         std::size_t memory_size;      public:         typedef std::size_t     size_type;         typedef T*              pointer;         typedef T               value_type;          PreAllocator(T* memory_ptr, std::size_t memory_size) : memory_ptr(memory_ptr), memory_size(memory_size) {}          PreAllocator(const PreAllocator& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};          template<typename U>         PreAllocator(const PreAllocator<U>& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};          template<typename U>         PreAllocator& operator = (const PreAllocator<U>& other) { return *this; }         PreAllocator<T>& operator = (const PreAllocator& other) { return *this; }         ~PreAllocator() {}           pointer allocate(size_type n, const void* hint = 0) {return memory_ptr;}         void deallocate(T* ptr, size_type n) {}          size_type max_size() const {return memory_size;} };  int main() {     int my_arr[100] = {0};     std::vector<int, PreAllocator<int>> my_vec(0, PreAllocator<int>(&my_arr[0], 100));     my_vec.push_back(1024);     std::cout<<"My_Vec[0]: "<<my_vec[0]<<"\n";     std::cout<<"My_Arr[0]: "<<my_arr[0]<<"\n";      int* my_heap_ptr = new int[100]();     std::vector<int, PreAllocator<int>> my_heap_vec(0, PreAllocator<int>(&my_heap_ptr[0], 100));     my_heap_vec.push_back(1024);     std::cout<<"My_Heap_Vec[0]: "<<my_heap_vec[0]<<"\n";     std::cout<<"My_Heap_Ptr[0]: "<<my_heap_ptr[0]<<"\n";      delete[] my_heap_ptr;     my_heap_ptr = nullptr; } 

Notice the difference between the two allocators! This will work with both heap buffers/arrays and stack buffer/arrays. It will also work with most containers. It is safer to use the Pre-C++11 version because it will be backwards compatible and work with more containers (ie: std::List).

You can just place the allocator in a header and use it as much as you want in any projects. It is good if you want to use SharedMemory or any buffer that is already allocated.

WARNING: DO NOT use the same buffer for multiple containers at the same time! A buffer can be reused but just make sure no two containers use it at the same time.

Example:

int my_arr[100] = {0}; std::vector<int, PreAllocator<int> > my_vec(PreAllocator<int>(&my_arr[0], 100)); std::vector<int, PreAllocator<int> > my_vec2(PreAllocator<int>(&my_arr[0], 100));  my_vec.push_back(1024); my_vec2.push_back(2048);  std::cout<<"My_Vec[0]: "<<my_vec[0]<<"\n"; std::cout<<"My_Arr[0]: "<<my_arr[0]<<"\n"; 

The output of the above is 2048! Why? Because the last vector overwrote the values of the first vector since they share the same buffer.

Answer 2

Yes, std::vector takes a custom allocator as a template parameter which can achieve what you want.