gogoogle-bigquerygoogle-api-go-client

Invalid Upload Request

I'm getting the following response, when trying to upload data to BigQuery using the Google API Client libraries for Go.

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "badRequest",
    "message": "Invalid Upload Request"
   }
  ],
  "code": 400,
  "message": "Invalid Upload Request"
 }
}

My job descriptor looks like this:

j := &bigquery.Job{
    Configuration: &bigquery.JobConfiguration{
        Load: &bigquery.JobConfigurationLoad{
            DestinationTable: &bigquery.TableReference{
                projectId,
                "xyz",
                name + "_" + yyyymmdd,
            },
            SkipLeadingRows: 1,
            FieldDelimiter:  "|",
            MaxBadRecords:   3,
            Schema: &bigquery.TableSchema{
                []*bigquery.TableFieldSchema{
                    {Name: "f1", Type: "STRING"},
                    {Name: "f2", Type: "STRING"},
                    {Name: "f3", Type: "STRING"},
                    {Name: "f4", Type: "STRING"},
                    {Name: "f5", Type: "STRING"},
                    {Name: "f6", Type: "STRING"},
                    {Name: "f7", Type: "STRING"},
                    {Name: "f8", Type: "STRING"},
                    {Name: "f9", Type: "STRING"},
                    {Name: "f10", Type: "STRING"},
                    {Name: "f11", Type: "STRING"},
                    {Name: "f12", Type: "STRING"},
                    {Name: "f13", Type: "STRING"},
                    {Name: "f14", Type: "STRING"},
                    {Name: "f15", Type: "STRING"},
                    {Name: "f16", Type: "STRING"},
                    {Name: "f17", Type: "STRING"},
                },
            },
        },
    },
}

FWIW, other calls using this library work fine, so I have ruled out oauth problems, etc.

EDIT: I captured the request before it went out. Please note that this is before the http.Client is able to add the oauth headers. The actual request is large, so I have shortened it and removed the identifiers.

POST /upload/bigquery/v2/projects/.../jobs HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5
Content-Length: 56369074
Content-Type: multipart/related; boundary=2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: application/json

{"configuration":{"load":{"destinationTable":{"projectId":"...","datasetId":"...","tableId":"..."},"skipLeadingRows":1,"fieldDelimiter":"|","maxBadRecords":3,"schema":{"fields":[{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"}]}}}}

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: text/plain; charset=utf-8

H0|H1|H2|H3|H4|H5|H6|H7|H8|H9|H10|H11|H12|H13|H14|H15|H16
f0_0|f0_1|f0_2|f0_3|f0_4|f0_5|f0_6|f0_7|f0_8|f0_9|f0_10|f0_11|f0_12|f0_13|f0_14|f0_15|f0_16
f1_0|f1_1|f1_2|f1_3|f1_4|f1_5|f1_6|f1_7|f1_8|f1_9|f1_10|f1_11|f1_12|f1_13|f1_14|f1_15|f1_16
# Three lines
# of comments
# in the trailer

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf--
Solution

  • It turns out that this was caused by a bug in the library that was not sending the required "uploadType" query parameter. See these diffs for a patch that fixes the bug.

    However this exposed another bug in the library that I am currently working with the author to fix. I will update this answer when that fix is in.

    EDIT: As of this patch, the bug referenced above is fixed.