Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Insert into BigQuery without a well defined struct

I'd like to insert any type of JSON directly into BigQuery, but haven't found any good way of doing this? All ways assume I have a well defined struct that I insert. It seems it was possible with the old deprecated api but not with the new "cloud.google.com/go/bigquery" package.

I'd like to have /api/table_name/insert be able to take any type of json and insert it into BigQuery as both the client and server knows the schema the endpoint should just forward it.

Thanks

like image 781
Nixarn Avatar asked Mar 24 '26 04:03

Nixarn


1 Answers

seems like this was replaced with ValueSaver which you can implement like:

type genericRecord map[string]bigquery.Value

func (rec genericRecord) Save() (map[string]bigquery.Value, string, error) {
    insertID := uuid.New().String()
    return rec, insertID, nil
}

var data []*genericRecord
json.Unmarshal(<YOUR JSON BYTE>, &data)

ctx := context.Background()
client, err := bigquery.NewClient(ctx, <YOUR PROJECT ID>)
ins := client.Dataset(<DATASET>).Table(<TABLE>).Inserter()
ins.Put(ctx, data)
like image 53
Itamar Lavender Avatar answered Mar 26 '26 16:03

Itamar Lavender



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!