How to use openSSL with memory BIOs and non blocking sockets

Question

I'm new to openSSL and I trying to figure out what the best / good solution is to create a https connection when using non blocking sockets, and libraries such as libevent, libev or libuv in combination with memory BIOs.

I'm trying to figure out how to manage openSSL calls/data and application data. In short my understanding of how a ssl client application should work is something like this:

• create SSL_CTX
• create a new socket connection (e.g. I'm using libuv)
• create two memory BIOs:
  • one is filled with data I receive from the server (readBio)
  • the other one is used to in the application code to read from. (writeBio)
• create a SSL* and set state to SSL_connect_state
• start the handshake process with SSL_do_handshake
• [loop] receive / send data

As I'm using libuv (but this could be any other async/non-blocking library), I have a read callback that gets called when data is received on the socket. When I have data which must be written to the socket, I pass this data into a write function of the library (in this uv_write()), but in between this I need to put the calls to SSL.

So after calling SSL_do_handshake(...), SSL stores some data into the writeBIO which I must read and pass into the socket. One question I was thinking about, how do I know that SSL stores data into this BIO, and secondly how do I know when I should send this over the socket.

After looking at some code, I figured out that I had to consume from the writeBIO after calling SSL_do_handshake(). But the next steps are not clear to me. After s ending the first bytes from the handshake the 'event' loop of libuv sets everything in motion; when new data arrives on the socket my 'onread() callback is called. But how do I handle this incoming data? (e.g. do I keep SSL state myself (<-- something which some people have advised me not do to)).

Although I've seen lots of examples which use blocking sockets and the core SSL functions to make a connection I haven't found a good clean/minimalistic example which shows how to use memory BIOs as a client.

I've pasted some code I'm using to test openSSL here: https://gist.github.com/3989091

Someone around who can describe the process of using async/non-blocking sockets and memory BIOs with SSL?

Thanks R

Answer 1

I've also put together a basic example of using memory BIO's with non-blocking sockets and a poll based event loop.

ssl_server_nonblock.c

I don' think it makes sense to post all the source code of that example here. But here is the outline of what the example code does.

Flow of encrypted & unencrypted bytes

This diagram shows how the read and write memory BIO's (rbio & wbio) are associated with the socket read and write respectively. On the inbound flow (data into the program) bytes are read from the socket and copied into the rbio via BIO_write. This represents the the transfer of encrypted data into the SSL object. The unencrypted data is then obtained through calling SSL_read. The reverse happens on the outbound flow to convey unencrypted user data into a socket write of encrypted data.

  +------+                                    +-----+
  |......|--> read(fd) --> BIO_write(rbio) -->|.....|--> SSL_read(ssl)  --> IN
  |......|                                    |.....|
  |.sock.|                                    |.SSL.|
  |......|                                    |.....|
  |......|<-- write(fd) <-- BIO_read(wbio) <--|.....|<-- SSL_write(ssl) <-- OUT
  +------+                                    +-----+

          |                                  |       |                     |
          |<-------------------------------->|       |<------------------->|
          |         encrypted bytes          |       |  unencrypted bytes  |