Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BigQuery row insertions failed with Go library

I'm getting this error "X row insertions failed" where X is a number of rows when I try to insert in bigquery.

I use this library "cloud.google.com/go/bigquery" in Golang

I attach the code here:

    u := table.Uploader()

    var inserts []*bigquery.StructSaver
    for _, insert := range value {
        aux := bigquery.StructSaver{Struct: insert, Schema: schema}
        inserts = append(inserts, &aux)
    }

    err := u.Put(ctx, inserts)
    if err != nil {
        fmt.Printf("%v\n", err)
    }

This doesn't happen in every try and I don't know what can produce it. Anyone got the same error?

like image 346
Javier Roda Avatar asked Jan 29 '17 13:01

Javier Roda


2 Answers

Using bigquery.PutMultiError I get the following properties where Message profiles the failure reason.

  • Location: column
  • Message: what failed
  • Reason: this can simply be invalid

Here is an example:

{Location: "speed"; Message: "Cannot convert value to integer (bad value):foobar"; Reason: "invalid"}

Here's an example using PutMultiError as mentioned by 1lann. Of note, Put can return a *errors.errorString or bigquery.PutMultiError so it's useful to check which you have.

err := u.Put(ctx, inserts)
if err != nil {
    if multiError, ok := err.(bigquery.PutMultiError); ok {
        for _, err1 := range multiError {
            for _, err2 := range err1.Errors {
                fmt.Println(err2)
            }
        }
    } else {
        fmt.Println(err)
    }
}
like image 200
Grokify Avatar answered Sep 30 '22 17:09

Grokify


One possible fix is to make sure the fields of your struct are exported (e.g. start with uppercase letters).

It's just a really bad error message, this is just one of the reasons you could be getting that error

like image 31
Oguz Yildiz Avatar answered Sep 30 '22 18:09

Oguz Yildiz