What is the correct way to emulate TCP connection through HTTP(s) proxy?

Question

One of my applications connect to a server module that is normally installed on different computer, sometimes on the internet. In some deployment scenarios users don't have direct internet access - only HTTP(s) proxy servers. So i need to teach my program to use HTTP proxy in order to emulate asynchronous TCP connection to server. There is a lot of info on the internet about this subject, and with HTTPS proxy it's really easy - just send "HTTP CONNECT" to port 443 of server app, send back response and you can send and receive binary data as you wish.

But some users have HTTPS disabled on proxy servers, so they only have HTTP. And there is a number of problems with HTTP due to proxy actively checking traffic, trying to cache it, accumulate etc. The 2 connections with one infinite "GET" HTTP request and one infinite HTTP response works, but different proxies offers different problems - for example, Microsoft IIS don't send small chunks of data instantly and tries to accumulate them :(.

So my question is: is it some well established technique to emulate full duplex TCP connection over HTTP proxy without HTTPS support? Maybe it's some open source or commercial implementations exists that i can use or buy? Any hints are welcome! I really don't want to create a solution that will work only on small number of proxies, so i need either already existing and tested implementation or good manual :).

Answer 1

This is a dupe of several other questions on SO.

Generally speaking, you cannot assume that a HTTP proxy will allow you to do TCP/IP streaming over a connection. This is something that has been discussed at great length in the HTML5 WebSockets working group.

In some cases, you can make a HTTP request using the CONNECT verb asking the proxy to generate a "blind" bi-directional tunnel to a target server/port combo. However, the proxy may well refuse to do so for any target port other than 443 (to prevent exactly what you're trying to do), and the proxy MAY try to scan or otherwise alter the traffic you send to it.

A SOCKS proxy, in contrast, is designed to do pretty much exactly what you're trying to do. But SOCKS proxies are relatively uncommon.