Menu
I would like to use the R package httr to access the EC2 services via their API. But I am a little bit unsure how to get started as it does not fall into the usual authentication format of "Oauth2.0" in which you have the usual: key, secret, token and signature system. I think EC2 uses the "signature version 2" method, but I am unclear as to how that works.
Looking at the documentation that EC2 provides with regard to making query requests at http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using-query-api.html
I think I need the value for signature....but don't know how to get it
I have tried using some of the given commands using httr as shown below. I can adapt most of the parameters in the URL string to represent me and the things i want to do e.g AWSAccessKeyId, ImageId, endpoint and the Action etc....but just dont know where to go about getting the signature value.
Also in some of the examples given, they dont seem to provide the secret access key either...
So the tried commands are as follows having altered some of the values to represent me but got the following:
require(httr)
GET("https://ec2.amazonaws.com/
?Action=RunInstances
&ImageId=ami-60a54009
&MaxCount=3
&MinCount=1
&Placement.AvailabilityZone=us-east-1b
&Monitoring.Enabled=true
&AWSAccessKeyId=0GS7553JW74RRM612K02EXAMPLE
&Version=2012-10-01
&Expires=2010-10-10T12:00:00Z
&Signature=lBP67vCvGlDMBQ1dofZxg8E8SUEXAMPLE
&SignatureVersion=2
&SignatureMethod=HmacSHA256")
to which i get the response:
Response [http://aws.amazon.com/ec2/]
Status: 200
Content-type: text/html; charset=UTF-8
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link rel="icon" type="image/ico" href="//d36cz9buwru1tt.cloudfront.net/favicon.ico">
<link rel="shortcut icon" type="image/ico" href="//d36cz9buwru1tt.cloudfront.net/favicon.ico">
<meta name="description" content="Amazon Elastic Compute Cloud delivers scalable, pay-as-you-go compute capacity in the cloud. " /><meta name="keywords" content="" /> ...
Has anybody had any experience with the EC2 api and its authentication procedure and would it be easy enough to use R to be able to setup and run linux instances with AMIs chosen by me (that have R and other relevant packages loaded onto it), then to run a few R commands in those instances, and bring the output back?
Don't think its really related to my sessionInfo but just in case here it is:
sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] httr_0.2
loaded via a namespace (and not attached):
[1] digest_0.5.2 plyr_1.7.1 RCurl_1.95-1.1 stringr_0.6.1 tools_2.15.1
EDIT:
So in a further attempt to follow the documentation as suggested by @hadley this is what I tried and got...any more helpful hints on where im going wrong would be greatly appreciated... :
require(httr)
aws.key <- "xxxxxxx"
aws.secret <- "xxxxxxxxxxxx"
verb <- "GET"
zone <- "ec2.amazonaws.com"
func <- "DescribeImages"
ami.number <- "ami-xxxxxxxxx"
params <- list(paste0("ImageId.1=",ami.number),
"Version=2012-10-01",
"Expires=2012-11-20T12%3A00%3A00Z")
# adding in method and key parameters for creation of string to sign
orig.len.params <- length(params)
params.w.method.key <- params
params.w.method.key[[orig.len.params+1]] <- "SignatureVersion=2"
params.w.method.key[[orig.len.params+2]] <- "SignatureMethod=HmacSHA1"
params.w.method.key[[orig.len.params+3]] <- paste0("AWSAccessKeyId=",aws.key)
# String to sign (s2s)
s2s <- paste(c(paste0(verb,"\n",zone,"\n","/\n","AWSAccessKeyId=",aws.key),paste0("Action=",func),paste(sort(unlist(params.w.method.key)),collapse="&")),collapse="&")
# Signature(sig)
sig <- hmac_sha1(aws.secret, s2s)
# adding in signature, method and key parameters for signed request url generation
params.w.sig.method.key <- params
params.w.sig.method.key[[orig.len.params+1]] <- paste0("Signature=",sig)
params.w.sig.method.key[[orig.len.params+2]] <- "SignatureVersion=2"
params.w.sig.method.key[[orig.len.params+3]] <- "SignatureMethod=HmacSHA1"
params.w.sig.method.key[[orig.len.params+4]] <- paste0("AWSAccessKeyId=",aws.key)
# Signed request (sr)
sr <- paste(c(paste0("https://",zone,paste0("?Action=",func)),paste(unlist(params.w.sig.method.key),collapse="&")),collapse="&")
# GET signed request
GET(sr)
to which I get the response:
Response [https://ec2.amazonaws.com?Action=DescribeImages&ImageId.1=[ami.number.from.before]&Version=2012-10-01&Expires=2012-11-20T12%3A00%3A00Z&Signature=[sig.value.from.before]&SignatureVersion=2&SignatureMethod=HmacSHA1&AWSAccessKeyId=[aws.key.from.before]/]
Status: 401
Content-type:
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>5e10fb0b-f304-4677-9c64-98b4537c659a</RequestID></Response>
886
Here's my attempt at a step by step conversion of the algorithm to R code. In my experience you really want to do each step separately so that you can check at each stage that the results are correct.
require("httr")
require("RCurl")
require("stringr")
# 0: get key and secret from envvars, and set up request parameters
aws.key <- Sys.getenv("AWS_KEY")
aws.secret <- Sys.getenv("AWS_SECRET_KEY")
verb <- "GET"
zone <- "ec2.amazonaws.com"
ami.number <- "ami-xxxxxxxxx"
params <- list(
Action = "DescribeImages",
ImageId.1 = ami.number,
Version = "2012-10-01",
Expires = "2012-11-20T12:00:00Z",
SignatureVersion = 2,
SignatureMethod = "HmacSHA1",
AWSAccessKeyId = aws.key)
# 1a: Sort the UTF-8 query string components by parameter name
params <- params[order(names(params))]
# 1b: URL encode the parameter name and values
params_e <- lapply(params, curlEscape)
names(params_e) <- curlEscape(names(params_e))
params_str <- str_c(names(params_e), "=", unlist(params_e), collapse = "&")
params_str <- gsub("%2E",".",gsub("%2D","-",params_str))
# 2: Create the string to sign
string_to_sign <- str_c(
toupper(verb), "\n",
tolower(zone), "\n",
"/", "\n",
params_str
)
# 3: Calculate an RFC 2104-compliant HMAC
# 4: Convert the resulting value to base64.
hmac <- hmac_sha1(aws.secret, string_to_sign)
params$Signature <- hmac
GET(paste0("https://",zone),query=params)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
