How to POST multipart/related content with httr (for Google Drive API)

Question

I got simple file uploads to Google Drive working using httr. The problem is that every document is uploaded as "untitled", and I have to PATCH the metadata to set the title. The PATCH request occasionally fails.

According to the API, I ought to be able to do a multipart upload, allowing me to specify the title as part of the same POST request that uploads the file.

res<-POST(
  "https://www.googleapis.com/upload/drive/v2/files?convert=true",
  config(token=google_token),
  body=list(y=upload_file(file))
)
id<-fromJSON(rawToChar(res$content))$id
if(is.null(id)) stop("Upload failed")
url<-paste(
  "https://www.googleapis.com/drive/v2/files/",
  id,
  sep=""
)
title<-strsplit(basename(file), "\\.")[[1]][1]
Sys.sleep(2)
res<-PATCH(url,
  config(token=google_token),
  body=paste('{"title": "',title,'"}', sep = ""),
  add_headers("Content-Type" = "application/json; charset=UTF-8")
)
stopifnot(res$status_code==200)
cat(id)

What I'd like to do is something like this:

res<-POST(
  "https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart&convert=true",
  config(token=google_token),
  body=list(y=upload_file(file),
            #add_headers("Content-Disposition" = "text/json"),
            json=toJSON(data.frame(title))
  ),
  encode="multipart",
  add_headers("Content-Type" = "multipart/related"),
  verbose()
)

The output I get shows that the content encoding of the individual parts is wrong, and it results in a 400 error:

-> POST /upload/drive/v2/files?uploadType=multipart&convert=true HTTP/1.1
-> User-Agent: curl/7.19.7 Rcurl/1.96.0 httr/0.6.1
-> Host: www.googleapis.com
-> Accept-Encoding: gzip
-> Accept: application/json, text/xml, application/xml, */*
-> Authorization: Bearer ya29.ngGLGA9iiOrEFt0ycMkPw7CZq23e6Dgx3Syjt3SXwJaQuH4B6dkDdFXyIC6roij2se7Fs-Ue_A9lfw
-> Content-Length: 371
-> Expect: 100-continue
-> Content-Type: multipart/related; boundary=----------------------------938934c053c6
-> 
<- HTTP/1.1 100 Continue
>> ------------------------------938934c053c6
>> Content-Disposition: form-data; name="y"; filename="db_biggest_tables.csv"
>> Content-Type: application/octet-stream
>> 

>> table    rows    DATA    idx total_size  idxfrac

>> 
>> ------------------------------938934c053c6
>> Content-Disposition: form-data; name="json"
>> 
>> {"title":"db_biggest_tables"}
>> ------------------------------938934c053c6--

<- HTTP/1.1 400 Bad Request
<- Vary: Origin
<- Vary: X-Origin
<- Content-Type: application/json; charset=UTF-8
<- Content-Length: 259
<- Date: Fri, 26 Jun 2015 18:50:38 GMT
<- Server: UploadServer
<- Alternate-Protocol: 443:quic,p=1
<- 

Is there any way to set the content encoding properly for individual parts? The second part should be "text/json", for example.

I have been through R documentation, Hadley's httr project pages at Github, this site and some general googling. I can't find any examples of how to do a multipart upload and set content-encoding.

Answer 1

You shoud be able to do this using curl::form_file or its alias httr::upload_file. See also the curl vignette. Following the example from the Google API doc:

library(httr)

media <- tempfile()
png(media, with = 800, height = 600)
plot(cars)
dev.off()

metadata <- tempfile()
writeLines(jsonlite::toJSON(list(title = unbox("My file"))), metadata)

#post
req <- POST("https://httpbin.org/post",
  body = list(
    metadata = upload_file(metadata, type = "application/json; charset=UTF-8"),
    media = upload_file(media, type = "image/png")
  ),
  add_headers("Content-Type" = "multipart/related"),
  verbose()
)

unlink(media)
unlink(metadata)

The only difference here is that curl will automatically add a Content-Disposition header for each file, which is required for multipart/form-data but not for multipart/related. The server will probably just ignore this redundant header in this case.

For now there is no way to accomplish this without writing the content to a file. Perhaps we could add something like that in a future version of httr/curl, although this has not come up before.