how to convert std::vector<float> to a tensor without copy in tensorflow in c++?

Question

In c++, a multidimensional matrix is stored in std::vector<float>. I need to use it in tensorflow, which uses tensors. The conversion from a std::vector to a tensor seems not obvious. There is a c_api which convert a vector to a TF_Tensor instead of Tensor. std::copy also works, but I want to perform a conversion without copy.

Answer 1

Tensorflow now has a way to do this in the C++ API by providing your own tensorflow::TensorBuffer and using the following constructor:

#include <tensorflow/core/framework/tensor.h>
#include <tensorflow/core/framework/types.pb.h>
...
tensorflow::Tensor(tensorflow::DataType type, const TensorShape & shape, TensorBuffer *buf)

Since tensorflow::TensorBuffer is an abstract class, you'll need to subclass it and implement a few methods yourself (that said, it's fairly easy to do). One thing to note: notice how we have OwnsMemory() returning false. If you want to use manual memory management (malloc/free or new/delete), you can set this to true and then override the destructor yourself. That said, since you're using a vector I'd just set it to false and take care to not have the buffer go out of scope. When it does, vector will free its own internal memory anyways.

eg;

class MyBuffer: public tensorflow::TensorBuffer {
    std::size_t len_;
  public:
    MyBuffer(void* data, std::size_t len): len_(len), tensorflow::TensorBuffer(data){}
    //returns how many bytes we have in our buffer
    std::size_t size() const override {return len_;};
    //needed so TF knows this isn't a child of some other buffer
    TensorBuffer* root_buffer() override { return this; }
    // Not actually sure why we need this, but it lets TF know where the memory for this tensor came from
    void FillAllocationDescription(tensorflow::AllocationDescription* proto) const override{};
    // A value of false indicates this TensorBuffer does not own the underlying data
    bool OwnsMemory() const override { return false; }
}

Then, you just need to provide the correct tensorflow::DataType (eg; tensorflow::DT_FLOAT32) and a tensorflow::TensorShape (you can just instantiate it and add each dimension using <TensorShape>.addDim(<the dimension>). You could modify the above by storing the std::vector inside and then exposing the contents by using .data() and a void* cast to make a constructor for MyBuffer that takes in a vector. Or you could just do that yourself outside of MyBuffer.