Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

FlatBuffers: Write to and read from binary file?

I have basic knowledge of file streams in C++ and Google FlatBuffers. The Schema file is quite simple, also creating a buffer and reading from a buffer pointer. The thing that I don't get is how to save multiple buffers into a binary file, and then read that binary file to get any random buffer.

Here is a simple Schema with two arrays of floats:

table Car {
    field_a:[float];
    field_b:[float];
}

.

A function for building the buffer (although without the file saving):

bool save_flatbuf(string file_path, vector<double> vec_a, vector<double> vec_b) {
    flatbuffers::FlatBufferBuilder builder;

    auto vec_floats_a = builder.CreateVector(vec_a, vec_a.size());
    auto vec_floats_b = builder.CreateVector(vec_b, vec_b.size());

    auto mloc = CreateCar(builder, &vec_floats_a, &vec_floats_b);

    builder.Finish(mloc);

    // How to save it into a binary file with a list of "Cars"?
}

.

And a function for reading the buffer after it was read from the binary file (without the file reading):

bool read_flatbuf(string file_path) {

    // How to get the buffer pointer to a "Car" from a binary file with a "list of Cars" ? .

    vector<double> final_vec_a;
    vector<double> final_vec_b;

    auto car = GetCar(buffer_pointer);

    auto fa = car->field_a();
    auto fb = car->field_b();

    final_vec_a.resize(fa->size());
    for (int i = 0; i < fa->size(); i++) {
        final_vec_a[i] = fa->Get(i);
    }

    final_vec_b.resize(fb->size());
    for (int i = 0; i < fb->size(); i++) {
        final_vec_b[i] = fb->Get(i);
    }
}

Not sure if the way to access the buffer's info is correct. For instance the way to get the length of the array fields.

Code examples for file interaction (write/read multiple buffers in one file) would be welcome.

like image 327
Davinish Avatar asked Nov 06 '14 17:11

Davinish


3 Answers

Quick reference to store buffer into binary file.

builder.Finish(mloc);
uint8_t *buf = builder.GetBufferPointer();
int size = builder.GetSize();

std::ofstream ofile("data.bin", std::ios::binary);
ofile.write((char *)buf, size);
ofile.close();

To read from file:

const std::string inputFile = "data.bin";
std::ifstream infile(inputFile, std::ios_base::binary);
std::vector<char> buffer( std::istreambuf_iterator<char>(infile),
                      std::istreambuf_iterator<char>());
like image 200
BlueDragon96 Avatar answered Oct 16 '22 09:10

BlueDragon96


My solution is adding additional size information.

for writer ::

for (item : flatbuffer_list) {
   int size = item.GetSize();
   write (file, &size, sizeof(size));
   write (file, item.GetBufferPointer(), item.GetSize());
}

for reader ::

while(!eof(file)) {
   int size;
   read (file, &size, sizeof(size));
   read (file, buffer, size);
   auto item = GetItem(buffer);
}
like image 42
user5814611 Avatar answered Oct 16 '22 07:10

user5814611


The best way to do this to add that list of cars to your schema:

table Garage {
  cars:[Car];
}

Then you can collect multiple car offsets (from CreateCar), call CreateVector on them, call CreateGarage on that, and then feed the result of that to Finish.

To read, start similarly from GetGarage(buffer_pointer).

like image 40
Aardappel Avatar answered Oct 16 '22 08:10

Aardappel