How to write a vector

Question

I am using the Go flatbuffers interface for the first time. I find the instructions sparse.

I would like to write a vector of uint64s into a table. Ideally, I would like to store numbers directly in a vector without knowing how many there are up front (I'm reading them from sql.Rows iterator). I see the generated code for the table has functions:

func DatasetGridAddDates(builder *flatbuffers.Builder, dates flatbuffers.UOffsetT) {
    builder.PrependUOffsetTSlot(2, flatbuffers.UOffsetT(dates), 0)
}
func DatasetGridStartDatesVector(builder *flatbuffers.Builder, numElems int) flatbuffers.UOffsetT {
    return builder.StartVector(8, numElems, 8)
}

Can I first write the vector using (??), then use DatasetGridAddDates to record the resulting vector in the containing "DatasetGrid" table?

Answer 1

(caveat: I have not heard of FlatBuffers prior to reading your question)

If you do know the length in advance, storing a vector is done as explained in the tutorial:

name := builder.CreateString("hello")

q55310927.DatasetGridStartDatesVector(builder, len(myDates))
for i := len(myDates) - 1; i >= 0; i-- {
    builder.PrependUint64(myDates[i])
}
dates := builder.EndVector(len(myDates))

q55310927.DatasetGridStart(builder)
q55310927.DatasetGridAddName(builder, name)
q55310927.DatasetGridAddDates(builder, dates)
grid := q55310927.DatasetGridEnd(builder)
builder.Finish(grid)

Now what if you don’t have len(myDates)? On a toy example I get exactly the same output if I replace StartDatesVector(builder, len(myDates)) with StartDatesVector(builder, 0). Looking at the source code, it seems like the numElems may be necessary for alignment and for growing the buffer. I imagine alignment might be moot when you’re dealing with uint64, and growing seems to happen automatically on PrependUint64, too.

So, try doing it without numElems:

q55310927.DatasetGridStartDatesVector(builder, 0)
var n int
for rows.Next() { // use ORDER BY to make them go in reverse order
    var date uint64
    if err := rows.Scan(&date); err != nil {
        // ...
    }
    builder.PrependUint64(date)
    n++
}
dates := builder.EndVector(n)

and see if it works on your data.