HTTPS request with Boost.Asio and OpenSSL

Question

I'm trying to read the ticker symbol at https://mtgox.com/api/0/data/ticker.php from my C++ application. I use Boost.Asio and OpenSSL because the service requires HTTPS.

Boost version: 1.47.0

OpenSSL: 1.0.0d [8 Feb 2011] Win32

For the application; I took the example from http://www.boost.org/doc/libs/1_47_0/doc/html/boost_asio/example/ssl/client.cpp to get started and modified it as follows:

This is where I want to connect to:

boost::asio::ip::tcp::resolver::query query("mtgox.com", "443");

I set verification to none because the handshake fails otherwise. I'm not sure if this is a problem with mtgox or that this implementation is really strict because when I print the certificate to the screen it looks legit (and chrome has no problem with it when visiting the ticker page).

socket_.set_verify_mode(boost::asio::ssl::context::verify_none);

This is the request I send:

std::stringstream request_;

request_ << "GET /api/0/data/ticker.php HTTP/1.1\r\n";
request_ << "Host: mtgox.com\r\n";
request_ << "Accept-Encoding: *\r\n";
request_ << "\r\n";

boost::asio::async_write(socket_, boost::asio::buffer(request_.str()), boost::bind(&client::handle_write, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred));

(full code: http://pastebin.com/zRTTqZVe)

I run into the following error:

Connection OK!
Verifying:
/C=IL/O=StartCom Ltd./OU=Secure Digital Certificate Signing/CN=StartCom Certification Authority
Sending request: 
GET /api/0/data/ticker.php HTTP 1.1
Host: mtgox.com
Accept-Encoding: *

Sending request OK!

Read failed: An existing connection was forcibly closed by the remote host

Am I going in the right direction with this? The error message isn't really descriptive of the problem and I don't know which step I did wrong.

Update: I used cURL to see what went wrong:

curl --trace-ascii out.txt https://mtgox.com/api/0/data/ticker.php

(full output: http://pastebin.com/Rzp0RnAK) It fails during verification. When I connect with the "unsafe" parameter

curl --trace-ascii out.txt -k https://mtgox.com/api/0/data/ticker.php

(full output: http://pastebin.com/JR43A7ux)

everything works fine.

Fix:

1. I fixed the typo in the HTTP headers
2. I added a root certificate and turned SSL verification back on.

Answer 1

In short:

1. You send "HTTP 1.1" instead of "HTTP/1.1". That's surely enough to make the server refuse your request. There are other differences between your request and cURL's, you might need to change those params as well - even if they seem valid to me.

2. Maybe OpenSSL does not have the root certificate used by the server, unlike Chrome, and that's why verification is failing.

Details:

1. Given a working and non-working tool, you should always compare what's happening. Here you have cURL's output and your request - comparing them showed a number of differences; usually, even with an encrypted connection, you can use a powerful packet sniffer like Wireshark, which decodes as much information from the packets as possible. Here it would allow to see that the server is actually sending less packets (I expect); another possibility would have been that your client was not getting the data sent by the server (say because the client had some bug).

2. If I understand correctly, curl showed only why you needed to disable verification, right? The certificate looks valid for me on chrome as well, but the root certification authority is quite unknown; curl mentions the "CA cert", i.e. the certificate of the certification authority. The root certificate is trusted because it is already present in a certificate DB on the client - I think that Chrome might have a more complete DB than OpenSSL (which is used by both cURL and your program).