How do I correctly digitally sign a SAML2.0 AuthnRequest?

Question

I have a working SAML2.0 Single Sign On system built in coffeescript on express / nodejs.

I use SSOCircle to test my SSO and can successfully authenticate there with either HTTP-POST or HTTP-REDIRECT binding. So far so good.

Now I need to digitally sign my authentication request.

To do this I have used the nodejs xml-crytpo library. This results in SAML that looks like this:

<?xml version="1.0"?>
<samlp:AuthnRequest Version="2.0" ID="_1C8A2EFA-2644-4330-9A36-2B45547ECFAF" IssueInstant="2014-10-14T16:06:27.769Z" Destination="https://idp.ssocircle.com:443/sso/SSOPOST/metaAlias/ssocircle" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion">
    <saml:Issuer>http://app.localhost</saml:Issuer>
    <samlp:NameIDPolicy AllowCreate="true" />
    <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
        <SignedInfo>
            <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
            <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
            <Reference URI="#_1C8A2EFA-2644-4330-9A36-2B45547ECFAF">
                <Transforms>
                    <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
                </Transforms>
                <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                <DigestValue>m4qHiXM82TuxY31l6+QSECHEHc0=</DigestValue>
            </Reference>
        </SignedInfo>
        <SignatureValue>fps0I0Rp02qDK0BPTK7Lh+ ...</SignatureValue>
        <KeyInfo>
            <X509Data>MIICuDCCAaCgAwIBAgIQEVFtJk ...</X509Data>
        </KeyInfo>
    </Signature>
</samlp:AuthnRequest>

The certificate information contained in the digital signature has been removed for brevity but it comes from a pem file I generated locally (self-signed).

I also updated my Service Provider metadata supplied to SSOCircle to include the SAML attributes AuthnRequestsSigned and WantAssertionsSigned, like this:

<md:EntityDescriptor entityID="http://app.localhost" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata">
    <md:SPSSODescriptor AuthnRequestsSigned="true" WantAssertionsSigned="true" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
        ...
    </md:SPSSODescriptor>
</md:EntityDescriptor>

Sending my AuthnRequest without the digital signature works fine. When I include the digital signature it fails with a statuscode of 500 and a generic 'Invalid Request' message.

How do I correctly digitally sign my AuthnRequest so it works against SSOCircle?

Thanks.

Edit 1 - Add Cert Key to the SP Metadata

Following suggestions in the comments I have now added the appropriate KeyDescriptor attributes to my sp-metadata to include my self-signed certificate's public-key.

<md:EntityDescriptor entityID="http://app.localhost" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata">
    <md:SPSSODescriptor AuthnRequestsSigned="true" WantAssertionsSigned="true" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
        <md:KeyDescriptor use="signing">
            <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:X509Data>
                    <ds:X509SubjectName>CN=app.localhost</ds:X509SubjectName>
                    <ds:X509Certificate>MIICu ... wg==</ds:X509Certificate>
                </ds:X509Data>
            </ds:KeyInfo>
        </md:KeyDescriptor>
        <md:KeyDescriptor use="encryption">
            <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:X509Data>
                    <ds:X509Certificate>MIICu ... wg==</ds:X509Certificate>
                </ds:X509Data>
            </ds:KeyInfo>
        </md:KeyDescriptor>
        ...
</md:EntityDescriptor>

I have included two KeyDescriptor elements, one specifying the use = signing and the other specifying the use = encryption, although I think the latter is redundant (as I am running over TLS) and will be ignored by the IdP.

This does not work and I get the same error:

Reason: The SAML Request is invalid.

Edit 2 - Self Signed Cerificates

Theory: I cannot use a self-signed certificate. Instead I must use a certificate that is known to and trusted by SSOCircle

Result: This was disproved. You can use a self-signed cert just as long as the correct public key is included in the sp-metadata.

Edit 3 - Added Correct Transform Algorithm

I have now updated my signature to specify the #enveloped-signature transformation algorithm.

        <Transforms>
            <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
        </Transforms>

Also I found that the correct structure for the included x509 certificate public key was the following (note the inner X509Certificate element):

<KeyInfo>
    <X509Data>
        <X509Certificate>MIID...MMxZ</X509Certificate>
    </X509Data>
</KeyInfo>

I am now using the TestShib site to run against as this allows access to the IdP logs which provides more detailed and useful information.

Edit 4 - Swapping the SAML Element Order to Match Schema

According to answer given by @Hos (see below) the order of the SAML elements must match the Schema. Therefore the signature element must appear directly after the issuer element, like this:

<samlp:AuthnRequest Version="2.0" ID="_782F6F1E-1B80-4D7D-B26C-AC85030D9300" IssueInstant="2014-10-28T11:45:49.412Z" Destination="https://idp.ssocircle.com:443/sso/SSOPOST/metaAlias/ssocircle" xmlns:samlp="urn:oasis:names:tc:SAML:2.0:protocol" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion">
    <saml:Issuer>http://app.localhost9de83841</saml:Issuer>
    <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
        <SignedInfo>
            <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
            <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
            <Reference URI="#_782F6F1E-1B80-4D7D-B26C-AC85030D9300">
                <Transforms>
                    <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
                </Transforms>
                <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                <DigestValue>pjXEtbAFMJA3hWhD/f6lxJTshTg=</DigestValue>
            </Reference>
        </SignedInfo>
        <SignatureValue>FY1...Qg==</SignatureValue>
        <KeyInfo>
            <X509Data>
                <X509Certificate>MIID...MMxZ</X509Certificate>
            </X509Data>
        </KeyInfo>
    </Signature>
    <samlp:NameIDPolicy AllowCreate="true" />
</samlp:AuthnRequest>

This still fails SSOCircle with an opaque 500 error. However TestShib shows the following log:

11:02:37.292 - DEBUG [edu.internet2.middleware.shibboleth.common.security.MetadataPKIXValidationInformationResolver:631] - Write lock over cache acquired
11:02:37.292 - DEBUG [edu.internet2.middleware.shibboleth.common.security.MetadataPKIXValidationInformationResolver:634] - Added new PKIX info to entity cache with key: [http://app.localhost9de83841,{urn:oasis:names:tc:SAML:2.0:metadata}SPSSODescriptor,urn:oasis:names:tc:SAML:2.0:protocol,SIGNING]
11:02:37.292 - DEBUG [edu.internet2.middleware.shibboleth.common.security.MetadataPKIXValidationInformationResolver:637] - Write lock over cache released
11:02:37.293 - WARN [edu.internet2.middleware.shibboleth.idp.profile.saml2.SSOProfileHandler:406] - Message did not meet security requirements
org.opensaml.ws.security.SecurityPolicyException: **Validation of protocol message signature failed**

As I read this, the public key used to validate the signature has been extracted from the sp-metadata:

Added new PKIX info to entity cache with key: [http://app.localhost9de83841,{urn:oasis:names:tc:SAML:2.0:metadata}SPSSODescriptor,urn:oasis:names:tc:SAML:2.0:protocol,SIGNING]

However the actual validation has failed, presumably because the digest values do not match. Which means the digest being calculated by xmlcrypto is different from the digest calculated by ShibTest. This can surely only be for one of two reasons:

• The algorithms used by xmlcrypto and TestShib to generate the digest are somehow different
• The public keys do not match.

A minute examination of the keys show they are the same. So here is my code in case you can spot anything.

Note: This code relies on the nodejs xmlbuilder and xml-crytpo libraries)

getSamlRequest:(idpUrl, requestId, next)->
    request = @xmlbuilder.create
        'samlp:AuthnRequest':
            '@xmlns:samlp':'urn:oasis:names:tc:SAML:2.0:protocol'
            '@xmlns:saml': 'urn:oasis:names:tc:SAML:2.0:assertion'
            '@Version': '2.0'
            '@ID': requestId
            '@IssueInstant': (new Date()).toISOString()
            '@Destination': idpUrl
            'saml:Issuer': '@@spEntityId'
        ,null
        ,headless: true
    request.comment 'insert-signature'
    request.element 'samlp:NameIDPolicy':
                        '@AllowCreate': 'true'
    saml = request.end()
    #Your self-signed pem file that contains both public and private keys. 
    #The public key must also be included in your sp-metadata
    certFilePath = "certs/sp-certificate.pem"
    @fs.readFile certFilePath, (err, certificate)=>
        signer = new @xmlcrypto.SignedXml()
        signer.signingKey = certificate
        signer.addReference "//*[local-name(.)='AuthnRequest']", ['http://www.w3.org/2000/09/xmldsig#enveloped-signature']
        signer.keyInfoProvider = new =>
            getKeyInfo: (key)=>
                public_key = /-----BEGIN CERTIFICATE-----([^-]*)-----END CERTIFICATE-----/g.exec(key)[1].replace /[\r\n|\n]/g, ''
                "<X509Data><X509Certificate>#{public_key}</X509Certificate></X509Data>"
        signer.computeSignature saml
        signature = signer.getSignatureXml()
        signed = saml.replace '<!-- insert-signature -->', signature
        return next null, signed

Edit 5 - Trace of Requests

Below is a trace of the requests made during the SAML handshake with SSOCircle. The output was generated by the nodejs request module in conjunction with the request-debug module.

I have included some clarification notes below this trace output.

{ request: 
   { debugId: 17,
     uri: 'http://localhost:8082/security/sso/subscription/dev2/096b75a2-4a55-4eec-83c8-d2509e948b07',
     method: 'GET',
     headers: { host: 'localhost:8082' } } }
{ response: 
   { debugId: 17,
     headers: 
      { 'transfer-encoding': 'chunked',
        'content-type': 'application/json; charset=utf-8',
        server: 'Microsoft-HTTPAPI/2.0',
        'access-control-allow-origin': '*',
        'access-control-allow-headers': 'origin, x-requested-with, accept, content-type',
        'access-control-allow-methods': 'GET, POST, PATCH, PUT, DELETE',
        date: 'Wed, 05 Nov 2014 16:55:49 GMT' },
     statusCode: 200,
     body: '{"RequestId":"_F7DDDD24-32C6-420E-A550-95872D30033B","SubscriptionName":"Develop-2","SubscriptionURL":"dev2","IdentityProviderCertificateFile":"ssocircle.cer","IdentityProviderDestinationUrl":"https://idp.ssocircle.com:443/sso/SSOPOST/metaAlias/ssocircle","SAMLBinding":"HttpPost"}' } }
{ request: 
   { debugId: 18,
     uri: 'https://idp.ssocircle.com:443/sso/SSOPOST/metaAlias/ssocircle',
     method: 'POST',
     headers: 
      { host: 'idp.ssocircle.com:443',
        'content-type': 'application/x-www-form-urlencoded; charset=utf-8',
        'content-length': 3492 },
     body: 'SAMLRequest=PHNhbWxwOkF1dGhuUmVxdWVzdCB4bWxuczpzYW1scD0idXJuOm9hc2lzOm5hbWVzOnRjOlNBTUw6Mi4wOnByb3RvY29sIiB4bWxuczpzYW1sPSJ1cm46b2FzaXM6bmFtZXM6dGM6U0FNTDoyLjA6YXNzZXJ0aW9uIiBWZXJzaW9uPSIyLjAiIElEPSJfRjdEREREMjQtMzJDNi00MjBFLUE1NTAtOTU4NzJEMzAwMzNCIiBJc3N1ZUluc3RhbnQ9IjIwMTQtMTEtMDVUMTY6NTU6NDkuODkwWiIgRGVzdGluYXRpb249Imh0dHBzOi8vaWRwLnNzb2NpcmNsZS5jb206NDQzL3Nzby9TU09QT1NUL21ldGFBbGlhcy9zc29jaXJjbGUiPjxzYW1sOklzc3Vlcj5odHRwOi8vYXBwLmxvY2FsaG9zdDlkZTgzODQxPC9zYW1sOklzc3Vlcj48U2lnbmF0dXJlIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwLzA5L3htbGRzaWcjIj48U2lnbmVkSW5mbz48Q2Fub25pY2FsaXphdGlvbk1ldGhvZCBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvMTAveG1sLWV4Yy1jMTRuIyIgLz48U2lnbmF0dXJlTWV0aG9kIEFsZ29yaXRobT0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnI3JzYS1zaGExIiAvPjxSZWZlcmVuY2UgVVJJPSIjX0Y3RERERDI0LTMyQzYtNDIwRS1BNTUwLTk1ODcyRDMwMDMzQiI%2BPFRyYW5zZm9ybXM%2BPFRyYW5zZm9ybSBBbGdvcml0aG09Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvMDkveG1sZHNpZyNlbnZlbG9wZWQtc2lnbmF0dXJlIiAvPjwvVHJhbnNmb3Jtcz48RGlnZXN0TWV0aG9kIEFsZ29yaXRobT0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnI3NoYTEiIC8%2BPERpZ2VzdFZhbHVlPjBLdUtNVG1Zc29Wb1BlTXhtdDhuNml2M3RxZz08L0RpZ2VzdFZhbHVlPjwvUmVmZXJlbmNlPjwvU2lnbmVkSW5mbz48U2lnbmF0dXJlVmFsdWU%2BY1htVFJPWVBSWFZPNHpWSkxURXBBSWd0RTd1c2NsSG1QS0NFdTB0NXNzcmVYSjd1a0M4dWdGd2c4Zm1yWDZDenhib3FHNVl1enJkb2RWVVNsbU12bHZXMGpiaWtsQVBXMmtPNTcralB4TWV3UTdFdzJZdTJuRTZ2QkFhMUwxWGpsa0g0a1UrdlVWVEpFQnlsdXhOYjRQN0xoZnJTdnVrZDhWejJwbk5QTnJtY0tJZVp1LzFlZU8wWmRyRVZrdEY1REhVaFV0MEs5aFBhRXB5Z0xsYjVKYWhNZEttei9uQWk4OU04aTEyUTdrQ2hwb1UrcmhjYTQzaDBXdGk2SWI1N1lWRzgyMzE5MlNsdGR5UkZPdXl2bEJRL0FMSkdpN0hNYzRMRXVkQktOL3pxaUFpK1NZYm1ONVNuN0NucFJWbW13U3NJUElncWxCbmlrU2pIQzRQS1VBPT08L1NpZ25hdHVyZVZhbHVlPjxLZXlJbmZvPjxYNTA5RGF0YT48WDUwOUNlcnRpZmljYXRlPk1JSURnRENDQW1pZ0F3SUJBZ0lESUFTeE1BMEdDU3FHU0liM0RRRUJCUVVBTUM0eEN6QUpCZ05WQkFZVEFrUkZNUkl3RUFZRFZRUUtFd2xUVTA5RGFYSmpiR1V4Q3pBSkJnTlZCQU1UQWtOQk1CNFhEVEUwTVRBeE5qRTBOVFl3TlZvWERURTFNVEF4TmpFME5UWXdOVm93TmpFTE1Ba0dBMVVFQmhNQ1JFVXhFakFRQmdOVkJBb1RDVk5UVDBOcGNtTnNaVEVUTUJFR0ExVUVBeE1LWW1sdlpuSmhZM1JoYkRDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS0Y0dDhzWVZNdHRVZkZ3eTZneFpQVjVWbWtCMDZwNXNqck5vUjUxUXZISmZFMkgyTnVTa0Nxa2paVmFwb1FRMStUVTNlelloMmxNVGNYSjU1Y0t2d0lUTlEvQWlLckxjaG4wdGxYZXFIRXhIdXBvazRGd1hqc20xSitpZTBvOUc1UDlRNTFXelRjRXYxSFRBV2Rkak9OK3Zsd3d0YndTaWRtNFBkN3hxZDdvQkhXTjJJSExQNlpGVHRPVWNpdzI1K0xtRk90V3dHdU41c1pNWDV6RDNUc216Y3ZNMFQwUzF0SVlHamhpaWNnM2UrbmhkWXhjSVNvZ1B1NjNlSWswKzM2OFU1TkhnYlJ5SVRnR2tPMFdtTU9PNDhpbFlFcWlPR1E3d3FxeTdrMzZzRGNxZXdiV1lvejFnQzBieGJBeXI1d3ZZUDdnaEVHYktSVXRWeDExZzdVQ0F3RUFBYU9CbmpDQm16QUpCZ05WSFJNRUFqQUFNQ3dHQ1dDR1NBR0crRUlCRFFRZkZoMVBjR1Z1VTFOTUlFZGxibVZ5WVhSbFpDQkRaWEowYVdacFkyRjBaVEFkQmdOVkhRNEVGZ1FVcTFyMnltMkRXb1NoYWgyRm11ekxJOWZjdDdZd1FRWURWUjBqQkRvd09LRXlwREF3TGpFTE1Ba0dBMVVFQmhNQ1JFVXhFakFRQmdOVkJBb1RDVk5UVDBOcGNtTnNaVEVMTUFrR0ExVUVBeE1DUTBHQ0FncVlNQTBHQ1NxR1NJYjNEUUVCQlFVQUE0SUJBUUFIRW5xTlVKN2VaYkkvNmozZTNmK2tFY3BQQUs2L3dXS0hSL1k2TWt1TGZpZERPREJ4eDRPTThQL1MrNFV4QkZSYWtBNFRkWkRNbjdJT2dTZ2Z3elQ0RjFxS1cvTkJKMzRtcjdndi9Yc0gxTDdHMlBEZFdUdmdGN1N5aFpCck9rbVYyZy9KYWg2U2pBREdabGdPWGJuTTlHUjlNN1NpVDBUcmFkay90dU9Zc2pxd2NJaE40eTVwSU9MNnZlemJKQThIeWZUY1lpMGFZVXFyMGl4Qkw1WWh5VDA1Qk13SUhkdFFNNGhqNjdyRDV4ME9QcmVrcTg0MjRkL0RGWmV5QTRBNG04Z3BOL0VDTFF0NzN3ajlqMVRudjBLdmlLRGZOcWJ4NngvL1o3MnN6VVhpWldETWJrNnhvdHVOZTV6Yy9xcXdhSWxLZ2lnWDJvcGRvbU9nTU14WjwvWDUwOUNlcnRpZmljYXRlPjwvWDUwOURhdGE%2BPC9LZXlJbmZvPjwvU2lnbmF0dXJlPjxzYW1scDpOYW1lSURQb2xpY3kgQWxsb3dDcmVhdGU9InRydWUiLz48L3NhbWxwOkF1dGhuUmVxdWVzdD4%3D' } }
{ response: 
   { debugId: 18,
     headers: 
      { server: '"SSOCircle Web Server"',
        date: 'Wed, 05 Nov 2014 16:55:48 GMT',
        'content-type': 'text/html;charset=UTF-8',
        connection: 'close',
        'set-cookie': [Object],
        'transfer-encoding': 'chunked' },
     statusCode: 500,
     body: '\n\n\n\n<html><head><title>\n            Error Page\n            \n        </title>\n    <link href="/css/bx.css" rel="stylesheet" type="text/css" /></head>\n    <body>\n        <div id="myheader">\n            <h1><img src="/logo.png" alt="IDPee - Put your LOGO here" height="65" width="180">&nbsp;</h1>\n        </div>\n        <div id="mycontent">\n            <div id="mynav">\n                <ul>\n                \n                </ul>\n            </div>\n            <div id="mybox">\n                <p>\n                    \n                    <h2>Error occurred</h2>\n<p>\nReason: Invalid signature in Request.\n</p>\n\n                    \n                </p>\n            </div>\n            \n        </div>\n        <div id="myfooter">\n            \n            Copyright &copy; SSOCircle/IDPee.com\n            \n            \n        </div>\n    </body>\n</html>\n\n\n\n' } }
error:  message=Something went wrong whilst trying to automatically log you in. Please try again., name=SSOFailure, message=An error occurred whilst processing the SSO Request., name=SSORequestFailure, message=The IdP failed to successfully process the SAMLRequest and returned an statuscode of 500., name=SamlRequestInvalid, url=http://app.localhost/security/saml2/request

Request debugId= 17: This is the call made by my client into my nodejs proxy server that kicks of the SAML handshake.

ResponsedebugId= 17: A 200 OK response from the proxy server back to the client. The client now waits to be redirected once the SAML handshake concludes.

Request debugId= 18: The SAMLRequest is POSTed to the SSOCircle SAML endpoint

ResponsedebugId= 18: A 500 Internal Server Error response from SSOCircle plus an HTML error page payload.

Answer 1

Concerning Edit2: A self signed certificate is sufficient for SSOCircle as long as it was included into the service provider's metadata.

Concerning Edit3: Please move the signature element right after the issuer element. The request must comply to the schema:

<sequence>
<element ref="saml:Issuer" minOccurs="0"/>
<element ref="ds:Signature" minOccurs="0"/>
<element ref="samlp:Extensions" minOccurs="0"/>
</sequence>