Best practice for secure socket connection

Question

What is the best practice for a secure socket connection (without SSL).

I will be moving sensitive data (logins/passwords/accounts) across TCP Socket connection, and wondering if there is a good/fast way of Encrypting/Decrypting and avoiding malicious injection.

Answer 1

If you are allergic to SSL, use SSH. But the principles are identical. A key is exchanged through an asymmetric algorithm (RSA, Diffie-Hellman...) then used for symmetric encryption and MAC. The construction is subtle so it is highly recommended that you use an existing protocol such as SSL or SSH, where tricky details have been dealt with.

In both protocols, a primary phase occurs with mutual authentication. The "server" part (in the connection model, the "client" is whoever is initiating the connection, and the server is the other machine) needs to present its asymmetric key in a way which is "convincing enough" for the client (the client wants to be sure that it is talking to the right server, not somebody else who is impersonating the server). In SSL, that key is part of an X.509 certificate. When SSL is used by a Web browser, that certificate is deemed correct if the browser can validate it against a known set of "trust anchors", also known as "root certificates". Those anchors are managed by some organizations (e.g. Verisign) which made a deal with the browser maker; bottom-line is that if you want a certificate which can be validated against those root certificates, then you have to get it from one of the anchor managers, and they will not do it for free.

Note, though, that validating against a known set of trust anchors is a browser thing. Since this is for your own project, you could create your own trust anchor, and your own certificates. OpenSSL can help you with that. Even simpler, use a "dummy" certificate for the server, also known by the client, and the client simply verifies that the server sends the "expected" certificate.

The simpler model is actually what SSH does. In SSH, each client is supposed to maintain its own database of "known server keys" (in $HOME/.ssh/known_hosts on Unix implementations). The client will display ample warnings when a new server is contacted, and even bigger warnings if a known server is contacted but does not use the previously recorded key. For many limited-scale projects, this is an appropriate model. The point of X.509 certificates with trust anchors is to allow clients to securely identify many servers, even servers which did not exist yet when the client software was deployed. If you control both client and server than you do not need that.

What SSH implements by default seems to better match what you are after, but SSL implementations are more common (in particular, I believe there is one in .NET, ready to be used). What you have to do is to capture the "server certificate validation" part, and replace it with your own simple matching function "is that the exact byte-to-byte certificate I was expecting ?".

Answer 2

If you don't want to use SSL/TLS, then you have to encrypt your sensitive data separately. For instance, protocols like HTTP, POP3, SMTP, and IMAP use SASL mechanisms to send encoded/encrypted data over unsecure connections.