How Do I Call XML SOAP Service that Requires Signature from .Net Core?

Question

I realize that this question refers to old technology. I am calling a vendor system and have no ability to change the service. We are required to call an XML/SOAP WS and then sign the request. 10 years ago, I would have used something like Web Services Enhancements (WSE) 3.0 and moved right along. As it is today, I'm stuck at what to do in our .Net Core (.Net Standard 2.0) application.

I'm willing to use many kinds of solutions, including commercial ones. I looked at Chilkat, but it seemed like we'd be giving up too much to use it.

They do have a decent example of what I'm referring to, however.

Given a request like:

<?xml version="1.0" encoding="UTF8"?>
<SOAP-ENV:Envelope xmlns:SOAPENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header>
 <wsse:Security xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
            xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"
            xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
            xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" SOAP-ENV:mustUnderstand="1">
 <wsse:BinarySecurityToken
           EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary"
           ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509"
           wsu:Id="x509cert00">BASE64_CERT</wsse:BinarySecurityToken>
 </wsse:Security>
</SOAP-ENV:Header>
<SOAP-ENV:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="TheBody">
 <getVersion xmlns="http://msgsec.wssecfvt.ws.ibm.com"/>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

We'd like to be able to use a certificate and sign it like this:

<?xml version="1.0" encoding="UTF8" ?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
    <SOAP-ENV:Header>
        <wsse:Security xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" SOAP-ENV:mustUnderstand="1">
            <wsse:BinarySecurityToken EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509" wsu:Id="x509cert00">MIIDgzCCAmugAwIBAgIBADANBgkqhkiG9w0BAQUFADBcMRUwEwYDVQQDDAxUZXN0
IENvbXBhbnkxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYD
VQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwHhcNMTcwOTEzMDA1NTM1WhcN
MTgwOTEzMDA1NTM1WjBcMRUwEwYDVQQDDAxUZXN0IENvbXBhbnkxCzAJBgNVBAYT
AkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBXaWRn
aXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDiWRKl
x+88u4SKZnfCMraqMsfJCs6tcz3EjMYTWmRKhhUOE9pDkvZfv0mgF7pNHsTKvFRt
oVnEVQaZC5TlHNOGa2QWit9YuruWjW8VSaU4s9gR1/Cg9/Zc8Z0yUEDpsaVnwuoA
RpVzvzoRzPmTNpMNEcQ07aBjHP7OJrwyvcdqQA1BbfDVMmRmw1d+/i8tyR3cTyzl
/3TismN5nrmhGh/ZF75FFA/xDN7PbVYDPowiFnEVHiBrYh2mFTabRUnb7K4oLx+d
1L5x3Az299F/HYZlBenXpJLtnCL3+HY6qsGXVbzKjlKNqbXsmlzVkChu093weN/q
UvWO2883cEiXmdqxAgMBAAGjUDBOMB0GA1UdDgQWBBRsMy2bxsCKYyUYtTYz/zZb
z7Le0zAfBgNVHSMEGDAWgBRsMy2bxsCKYyUYtTYz/zZbz7Le0zAMBgNVHRMEBTAD
AQH/MA0GCSqGSIb3DQEBBQUAA4IBAQBnFQ+Sc3s8y79DTsA7CvvAFeG/zvWQiu8y
UM5LO1QcWeQQj29GMThqrY21dNfkynl7mZUMEeXKvwwzweFCc2odiUPHxoV1G4FE
tzNaZ8Ap9jye78YQ8SB8NPQwC7ovecfSqNflT4NMAThSuxpGp8Ugf7a24LXozLzL
bCRvG9sLGyRneZbfU8B43ELRLCkjzWR32N7D2pmKk4CEMiW0ScphU1JEHaimneMa
TFc63hNzKpuj7+BGv4ZuvB1j/Mbmz53PGgFKnGQHPb2TIvMxyB+lML5vE0Bm8YWt
P8DNyx11CCCdBdMWfeta6MjmmqcV5/YEq92c5O2Ql94tWFNLR6wQ</wsse:BinarySecurityToken>
            <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:SignedInfo>
                    <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                        <InclusiveNamespaces xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="wsse SOAP-ENV" />
                    </ds:CanonicalizationMethod>
                    <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
                    <ds:Reference URI="#TheBody">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>VhsSnaEAFsY0OYegKQh99v9csXg=</ds:DigestValue>
                    </ds:Reference>
                </ds:SignedInfo>
                <ds:SignatureValue>Ynp3H4rtzpXIh4TaVxkpEkS1bMCCu672aeCzUOzheNNfnpmLsCZz3+zQjMBbchPggCayC5ihpEdhRe3XvPXjPXXAgxDP4mic091QPmjHlmUcu8yqRKfxnPtD35nqaxDtCYw+jGIzj+ch094vA4RPCfY8JQnb1mpy1ZjjsMW8741CIh1epbsd/0bZt6tfINUQ37seg07yvLbCJZ/Zf+h8FlFryQk6lHTTeZl/GfQ9NlDBcShby3x8Hc1KwW++zFqEA7G783R9AYPYn3fWTOBhYk5gkgFc+HaPRLR/L0Bp7ZPbmOR/iZQ+HK4W672tTdN/R2GdN7/deV7QTp2DYK1Z8w==</ds:SignatureValue>
                <ds:KeyInfo>
                    <wsse:SecurityTokenReference>
                        <wsse:Reference URI="#x509cert00" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509" />
                    </wsse:SecurityTokenReference>
                </ds:KeyInfo>
            </ds:Signature>
        </wsse:Security>
    </SOAP-ENV:Header>
    <SOAP-ENV:Body xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="TheBody">
        <getVersion xmlns="http://msgsec.wssecfvt.ws.ibm.com" />
    </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Answer 1

I solved this by rolling our own soap envelopes, signing them and the piping it over HttpClient. WCF in .NET Core just couldn't get a result for us that worked with the various quirks of our third party service.

Here's the code, it should be easy enough to alter to your requirements:

// ...
private static HttpClient Client = new HttpClient(); // https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/
// ...

Uri uri = new Uri("https://thirdparty.com/service.svc");
X509Certificate2 cert = // from some store etc
var envelope = BuildEnvelope(cert);

using (HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, uri))
{
    request.Content = new StringContent(envelope, Encoding.UTF8, "application/soap+xml");
    using (HttpResponseMessage response = Client.SendAsync(request).Result)
    {
        if (response.IsSuccessStatusCode)
        {
            response.Content.ReadAsStringAsync().ContinueWith(task =>
            {
                string thirdparty_envelope = task.Result;
                XElement thirdparty_root = XElement.Parse(thirdparty_envelope);
                // etc
            }, TaskContinuationOptions.ExecuteSynchronously);
        }
    }
}


private string BuildEnvelope(X509Certificate2 certificate)
{
    string envelope = null;
    // note - lots of bits here specific to my thirdparty
    string cert_id = string.Format("uuid-{0}-1", Guid.NewGuid().ToString());
    using (var stream = new MemoryStream())
    {
        Encoding utf8 = new UTF8Encoding(false); // omit BOM
        using (var writer = new XmlTextWriter(stream, utf8))
        {
            // timestamp
            DateTime dt = DateTime.UtcNow;
            string now = dt.ToString("o").Substring(0, 23) + "Z";
            string plus5 = dt.AddMinutes(5).ToString("o").Substring(0, 23) + "Z";

            // soap envelope
            // <s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing" xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            writer.WriteStartDocument();
            writer.WriteStartElement("s", "Envelope", "http://www.w3.org/2003/05/soap-envelope");
            writer.WriteAttributeString("xmlns", "a", null, "http://www.w3.org/2005/08/addressing");
            writer.WriteAttributeString("xmlns", "u", null, "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd");

            writer.WriteStartElement("s", "Header", null);

            /////////////////
            //  saml guts  //
            /////////////////

            //<a:Action s:mustUnderstand="1">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue</a:Action>
            writer.WriteStartElement("a", "Action", null);
            writer.WriteAttributeString("s", "mustUnderstand", null, "1");
            writer.WriteString("http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue");
            writer.WriteEndElement(); //Action

            //<a:MessageID>urn:uuid:0cc426dd-35bf-4c8b-a737-7e2ae94bd44d</a:MessageID>
            string msg_id = string.Format("urn:uuid:{0}", Guid.NewGuid().ToString());
            writer.WriteStartElement("a", "MessageID", null);
            writer.WriteString(msg_id);
            writer.WriteEndElement(); //MessageID

            //<a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo>
            writer.WriteStartElement("a", "ReplyTo", null);
            writer.WriteStartElement("a", "Address", null);
            writer.WriteString("http://www.w3.org/2005/08/addressing/anonymous");
            writer.WriteEndElement(); //Address
            writer.WriteEndElement(); //ReplyTo

            writer.WriteStartElement("a", "To", "http://www.w3.org/2005/08/addressing");
            writer.WriteAttributeString("s", "mustUnderstand", null, "1");
            writer.WriteAttributeString("u", "Id", null, "_1");
            writer.WriteString("https://thirdparty.com/service.svc");
            writer.WriteEndElement(); //To

            //<o:Security xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" s:mustUnderstand="1">
            writer.WriteStartElement("o", "Security", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
            writer.WriteAttributeString("s", "mustUnderstand", null, "1");

            //<u:Timestamp u:Id="_0">
            writer.WriteStartElement("u", "Timestamp", null);
            writer.WriteAttributeString("u", "Id", null, "_0");

            //<u:Created>2018-02-08T15:03:13.115Z</u:Created>
            writer.WriteElementString("u", "Created", null, now);

            //<u:Expires>2018-02-08T15:08:13.115Z</u:Expires>
            writer.WriteElementString("u", "Expires", null, plus5);

            writer.WriteEndElement(); //Timestamp

            writer.WriteStartElement("o", "BinarySecurityToken", null);
            writer.WriteAttributeString("u", "Id", null, cert_id);
            writer.WriteAttributeString("ValueType", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3");
            writer.WriteAttributeString("EncodingType", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary");
            byte[] rawData = certificate.GetRawCertData();
            writer.WriteBase64(rawData, 0, rawData.Length);
            writer.WriteEndElement(); //BinarySecurityToken

            writer.WriteEndElement(); //Security
            writer.WriteEndElement(); //Header

            //<s:Body xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            writer.WriteStartElement("s", "Body", "http://www.w3.org/2003/05/soap-envelope");
            writer.WriteAttributeString("xmlns", "xsd", null, "http://www.w3.org/2001/XMLSchema");
            writer.WriteAttributeString("xmlns", "xsi", null, "http://www.w3.org/2001/XMLSchema-instance");

            // your 3rd-party soap payload goes here
            writer.WriteStartElement("???", "http://docs.oasis-open.org/ws-sx/ws-trust/200512");
            // ...                
            writer.WriteEndElement(); // 
            writer.WriteEndElement(); // Body


            writer.WriteEndElement(); //Envelope
        }

        // signing pass
        var signable = Encoding.UTF8.GetString(stream.ToArray());
                            XmlDocument doc = new XmlDocument();
        doc.LoadXml(signable);

        // see https://stackoverflow.com/a/6467877
        var signedXml = new SignedXmlWithId(doc);

        var key = certificate.GetRSAPrivateKey();
        signedXml.SigningKey = key;
        // these values may not be supported by your 3rd party - they may use e.g. SHA256 miniumum
        signedXml.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NTransformUrl;
        signedXml.SignedInfo.SignatureMethod = SignedXml.XmlDsigRSASHA1Url;

        // 
        KeyInfo keyInfo = new KeyInfo();
        KeyInfoX509Data x509data = new KeyInfoX509Data(certificate);
        keyInfo.AddClause(x509data);
        signedXml.KeyInfo = keyInfo;

        // 3rd party wants us to only sign the timestamp fragment- ymmv
        Reference reference0 = new Reference();
        reference0.Uri = "#_0";
        var t0 = new XmlDsigExcC14NTransform();
        reference0.AddTransform(t0);
        reference0.DigestMethod = SignedXml.XmlDsigSHA1Url;
        signedXml.AddReference(reference0);
        // etc

        // get the sig fragment
        signedXml.ComputeSignature();
        XmlElement xmlDigitalSignature = signedXml.GetXml();

        // modify the fragment so it points at BinarySecurityToken instead
        XmlNode info = null;
        for (int i = 0; i < xmlDigitalSignature.ChildNodes.Count; i++)
        {
            var node = xmlDigitalSignature.ChildNodes[i];
            if (node.Name == "KeyInfo")
            {
                info = node;
                break;
            }
        }
        info.RemoveAll();

        // 
        XmlElement securityTokenReference = doc.CreateElement("o", "SecurityTokenReference", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
        XmlElement reference = doc.CreateElement("o", "Reference", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
        reference.SetAttribute("ValueType", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3");
        // cert id                
        reference.SetAttribute("URI", "#" + cert_id);
        securityTokenReference.AppendChild(reference);
        info.AppendChild(securityTokenReference);

        var nsmgr = new XmlNamespaceManager(doc.NameTable);
        nsmgr.AddNamespace("o", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd");
        nsmgr.AddNamespace("s", "http://www.w3.org/2003/05/soap-envelope");
        var security_node = doc.SelectSingleNode("/s:Envelope/s:Header/o:Security", nsmgr);
        security_node.AppendChild(xmlDigitalSignature);

        envelope = doc.OuterXml;
    }

    return envelope;
}