Flash SecureSocket and RSA private key

Question

I am trying to communicate to a server through SSL. The PEM client file is consisted of a certificate and an rsa private key.

I managed to convert both the certificate and key to binary DER. I load the DER certificate to SecureSocket succesfully (with function addBinaryChainBuildingCertificate) but when I try to connect to the server I get a "principal mismatch" error. If I try to use the aforementioned function to load the DER key, I get a "wrong parameter" error.

I suppose the "principal mismatch" is because I haven't loaded the private key. But I see no function to load an RSA key to SecureSocket. Is there any solution to this? Do I need to communicate to the server with only a certificate but remove the key from the equation?

EDIT :

Code :

package {

    import flash.display.Sprite;
    import flash.net.SecureSocket;
    import flash.net.URLLoader;
    import flash.events.ProgressEvent;
    import flash.events.Event;
    import flash.events.IOErrorEvent;
    import flash.net.URLLoaderDataFormat;
    import flash.net.URLRequest;
    import flash.utils.ByteArray;

    public class TestSSL2 extends Sprite {

        private var mSocket:SecureSocket = new SecureSocket();

        private var certFile:String = "ca.der";
        private var keyFile:String = "key.der";

        private var cert:ByteArray;
        private var key:ByteArray;

        public function TestSSL2() {
            trace("SecureSocket.isSupported",SecureSocket.isSupported);

            var urlLoader:URLLoader = new URLLoader();
            urlLoader.addEventListener(Event.COMPLETE, certLoaded, false, 0, true);
            urlLoader.dataFormat = URLLoaderDataFormat.BINARY;
            urlLoader.load(new URLRequest(certFile));
        }
        private function certLoaded(e:Event):void {
            cert = (e.target as URLLoader).data;
            trace("certificate",cert.length);
            mSocket.addBinaryChainBuildingCertificate(cert, true);

            var urlLoader:URLLoader = new URLLoader();
            urlLoader.addEventListener(Event.COMPLETE, keyLoaded, false, 0, true);
            urlLoader.dataFormat = URLLoaderDataFormat.BINARY;
            urlLoader.load(new URLRequest(keyFile));
        }
        private function keyLoaded(e:Event):void {
            key = (e.target as URLLoader).data;
            trace("key",key.length);
            mSocket.addBinaryChainBuildingCertificate(key, true);

            mSocket.connect("127.0.0.1", 3000);
            mSocket.addEventListener(Event.CONNECT, socketConnected);
            mSocket.addEventListener(IOErrorEvent.IO_ERROR, onError);
            mSocket.addEventListener(ProgressEvent.SOCKET_DATA, socketData );
        }

        private function onError(error:IOErrorEvent):void {
            trace("ERROR!",error.text,":",mSocket.serverCertificateStatus);
        }

        private function socketConnected(e:Event):void {
            trace("Connected", e);
        }

        private function socketData(e:*):void {
            var data:String;
            data = mSocket.readUTFBytes(mSocket.bytesAvailable);
            trace(data);
        }
    }

}

Result :

SecureSocket.isSupported true
certificate 497
key 607
ArgumentError: Error #2004: One of the parameters is invalid.
    at flash.net::SecureSocket/addBinaryChainBuildingCertificate()
    at TestSSL2/keyLoaded()
    at flash.events::EventDispatcher/dispatchEventFunction()
    at flash.events::EventDispatcher/dispatchEvent()
    at flash.net::URLLoader/onComplete()

If I comment the line :

//mSocket.addBinaryChainBuildingCertificate(key, true);

I get :

SecureSocket.isSupported true
certificate 497
key 607
ERROR! Error #2031: Socket Error. URL: 127.0.0.1 : principalMismatch

Answer 1

Firstly:

The "principal mismatch" indicates that the common name of the certificate on the secured server does not match the DNS name that you are connecting to.

Considering that you are making a connect to localhost (127.0.0.1) there will most certainly be a mismatch. Flash sockets are particularly strict when it comes to making secure connections and there is no mechanism to override security features unlike other runtimes (e.g. .NET and Java). The following must be true:

1. The certificate common name mush match the DNS name (there is a relaxation here for 'star' certs i.e. a cert for *.bob.com is considered valid for mr.bob.com)
2. The certificate must be valid in terms of expiry and trust chain

Secondly:

You seem to have some misconception on how certificates work. You do not need to add any certificates using the addBinaryChainBuildingCertificate() method if the server certificate is issued by a trusted root authority i.e. the certificate that signed the server certificate is in the local trust store of the target device.

To illustrate:

• I have a certificate for this.is.awesome.com installed on my server and a DNS entry that resolves this.is.awesome.com to my server's IP Address
• This certificate is issued by the Entrust L1C certificate authority.
• The L1C certificate is in turn issued by the Entrust 2048 root authority.

On my PC I have the Entrust 2048 Root authority installed in my Trusted Root certificate store. However I do not have the L1C certificate installed. When I attempt to connect to this.is.awesome.com the connection will fail as the server certificate cannot be validated against the L1C authority.

If I add the DER encoded L1C certificate using addBinaryChainBuildingCertificate() then the connection will succeed. The server certificate will be validated against the L1C certificate which will in turn be validated against the 2048 Root certificate which is a trusted root.

To summarize:

Your connection issues seem to stem from your attempt to connect to localhost. Try adding an entry to your HOSTS file that maps the name on your certificate to 127.0.0.1 and then connecting to that name. If that fails check the issuer chain on your certificate and add the chain of issuers by calling addBinaryChainBuildingCertificate() once for each issuing certificate in the chain. The final or root certificate should be marked as such by passing true as the second parameter to addBinaryChainBuildingCertificate()