CloudKit Server-to-Server authentication

Question

Apple published a new method to authenticate against CloudKit, server-to-server. https://developer.apple.com/library/content/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/SettingUpWebServices.html#//apple_ref/doc/uid/TP40015240-CH24-SW6

I tried to authenticate against CloudKit and this method. At first I generated the key pair and gave the public key to CloudKit, no problem so far.

I started to build the request header. According to the documentation it should look like this:

X-Apple-CloudKit-Request-KeyID: [keyID]  
X-Apple-CloudKit-Request-ISO8601Date: [date]  
X-Apple-CloudKit-Request-SignatureV1: [signature]

• [keyID], no problem. You can find this in the CloudKit dashboard.
• [Date], I think this should work: 2016-02-06T20:41:00Z
• [signature], here is the problem...

The documentation says:

The signature created in Step 1.

Step 1 says:

Concatenate the following parameters and separate them with colons.
[Current date]:[Request body]:[Web Service URL]

I asked myself "Why do I have to generate the key pair?".
But step 2 says:

Compute the ECDSA signature of this message with your private key.

Maybe they mean to sign the concatenated signature with the private key and put this into the header? Anyway I tried both...

My sample for this (unsigned) signature value looks like:

2016-02-06T20:41:00Z:YTdkNzAwYTllNjI1M2EyZTllNDNiZjVmYjg0MWFhMGRiMTE2MjI1NTYwNTA2YzQyODc4MjUwNTQ0YTE5YTg4Yw==:https://api.apple-cloudkit.com/database/1/[iCloud Container]/development/public/records/lookup  

The request body value is SHA256 hashed and after that base64 encoded. My question is, I should concatenate with a ":" but the url and the date also contains ":". Is it correct? (I also tried to URL-Encode the URL and delete the ":" in the date).
At next I signed this signature string with ECDSA, put it into the header and send it. But I always get 401 "Authentication failed" back. To sign it, I used the ecdsa python module, with following commands:

from ecdsa import SigningKey  
a = SigningKey.from_pem(open("path_to_pem_file").read())  
b = "[date]:[base64(request_body)]:/database/1/iCloud....."  
print a.sign(b).encode('hex')

Maybe the python module doesn't work correctly. But it can generate the right public key from the private key. So I hope the other functions also work.

Has anybody managed to authenticate against CloudKit with the server-to-server method? How does it work correctly?

Edit: Correct python version that works

from ecdsa import SigningKey
import ecdsa, base64, hashlib  

a = SigningKey.from_pem(open("path_to_pem_file").read())  
b = "[date]:[base64(sha256(request_body))]:/database/1/iCloud....."  
signature = a.sign(b, hashfunc=hashlib.sha256, sigencode=ecdsa.util.sigencode_der)  
signature = base64.b64encode(signature)
print signature #include this into the header

Answer 1

The last part of the message

[Current date]:[Request body]:[Web Service URL]

must not include the domain (it must include any query parameters):

2016-02-06T20:41:00Z:YTdkNzAwYTllNjI1M2EyZTllNDNiZjVmYjg0MWFhMGRiMTE2MjI1NTYwNTA2YzQyODc4MjUwNTQ0YTE5YTg4Yw==:/database/1/[iCloud Container]/development/public/records/lookup

With newlines for better readability:

2016-02-06T20:41:00Z
:YTdkNzAwYTllNjI1M2EyZTllNDNiZjVmYjg0MWFhMGRiMTE2MjI1NTYwNTA2YzQyODc4MjUwNTQ0YTE5YTg4Yw==
:/database/1/[iCloud Container]/development/public/records/lookup

The following shows how to compute the header value in pseudocode

The exact API calls depend on the concrete language and crypto library you use.

//1. Date
//Example: 2016-02-07T18:58:24Z
//Pitfall: make sure to not include milliseconds
date = isoDateWithoutMilliseconds() 

//2. Payload
//Example (empty string base64 encoded; GET requests):
//47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
//Pitfall: make sure the output is base64 encoded (not hex)
payload = base64encode(sha256(body))  

//3. Path
//Example: /database/1/[containerIdentifier]/development/public/records/lookup
//Pitfall: Don't include the domain; do include any query parameter
path = stripDomainKeepQueryParams(url) 

//4. Message
//Join date, payload, and path with colons
message = date + ':' + payload + ':' + path

//5. Compute a signature for the message using your private key.
//This step looks very different for every language/crypto lib.
//Pitfall: make sure the output is base64 encoded.
//Hint: the key itself contains information about the signature algorithm 
//      (on NodeJS you can use the signature name 'RSA-SHA256' to compute a 
//      the correct ECDSA signature with an ECDSA key).
signature = base64encode(sign(message, key))

//6. Set headers
X-Apple-CloudKit-Request-KeyID = keyID 
X-Apple-CloudKit-Request-ISO8601Date = date  
X-Apple-CloudKit-Request-SignatureV1 = signature

//7. For POST requests, don't forget to actually send the unsigned request body
//   (not just the headers)

Answer 2

Extracting Apple's cloudkit.js implementation and using the first call from the Apple sample code node-client-s2s/index.js you can construct the following:

You hash the request body request with sha256:

var crypto = require('crypto');
var bodyHasher = crypto.createHash('sha256');
bodyHasher.update(requestBody);
var hashedBody = bodyHasher.digest("base64");

The sign the [Current date]:[Request body]:[Web Service URL] payload with the private key provided in the config.

var c = crypto.createSign("RSA-SHA256");
c.update(rawPayload);
var requestSignature = c.sign(key, "base64");

Another note is the [Web Service URL] payload component must not include the domain but it does need any query parameters.

Make sure the date value is the same in X-Apple-CloudKit-Request-ISO8601Date as it is in the signature. (These details are not documented completely, but is observed by looking through the CloudKit.js implementation).

A more complete nodejs example looks like this:

(function() {

const https = require('https');
var fs = require('fs');
var crypto = require('crypto');

var key = fs.readFileSync(__dirname + '/eckey.pem', "utf8");
var authKeyID = 'auth-key-id';

// path of our request (domain not included)
var requestPath = "/database/1/iCloud.containerIdentifier/development/public/users/current";

// request body (GET request is blank)
var requestBody = '';

// date string without milliseconds
var requestDate = (new Date).toISOString().replace(/(\.\d\d\d)Z/, "Z");

var bodyHasher = crypto.createHash('sha256');
bodyHasher.update(requestBody);
var hashedBody = bodyHasher.digest("base64");

var rawPayload = requestDate + ":" + hashedBody + ":" + requestPath;

// sign payload
var c = crypto.createSign("sha256");
c.update(rawPayload);
var requestSignature = c.sign(key, "base64");

// put headers together
var headers = {
    'X-Apple-CloudKit-Request-KeyID': authKeyID,
    'X-Apple-CloudKit-Request-ISO8601Date': requestDate,
    'X-Apple-CloudKit-Request-SignatureV1': requestSignature
};

var options = {
    hostname: 'api.apple-cloudkit.com',
    port: 443,
    path: requestPath,
    method: 'GET',
    headers: headers
};

var req = https.request(options, (res) => {
   //... handle nodejs response
});

req.end();

})();

This also exists as a gist: https://gist.github.com/jessedc/a3161186b450317a9cb5

On the command line with openssl (Updated)

The first hashing can be done with this command:

openssl sha -sha256 -binary < body.txt | base64

To sign the second part of the request you need a more modern version of openSSL than what OSX 10.11 comes with and use the following command:

/usr/local/bin/openssl dgst -sha256WithRSAEncryption -binary -sign ck-server-key.pem raw_signature.txt | base64

Thanks to @maurice_vB below and on twitter for this info