HTTPs proxy server only works in SwitchOmega

Question

I did quite a lot search and pratical trials before asking this question.

Long story:

I found a (non-English)tutorial about how to write a http proxy with Node.js.

So far what I've known and tried:

• A HTTP proxy can handle both HTTP request and HTTPS request, but in different ways. It handles HTTP request by reading the client's request and make a new request to the target and return the response to the client. As for HTTPS request, it's dealt with a HTTP Tunnel.

• The SSL proxy field in Firefox proxy settings and the Secure field in IE proxy settings (Windows) are all about setting the HTTP Tunnel. If a SSL proxy or Secure proxy is set, when a brower wants to connect to a https site, it sends a CONNECT request instead of an ordinary request.

Problems:

The CONNECT request is plain text, so firewalls can see what host I want to connect to and cut the connection. So I was thinking whether I can use https to talk to the proxy server from the very beginning. I read all related posts, but couldn't find an answer directly talking about this. And some answers also say "There's no such thing as a https proxy server".

But the tutorial says this can be done (HTTPS between client and proxy server and nothing else changes）. So I give it try. I changed the server into https with my website's certificate. But eventually it only works with Proxy SwitchOmega in Chrome. It doesn't work in traditional settings like in Firefox proxy or IE proxy settings.

Proxy SwitchOmega setting:

Scheme|Protocol|Server|Port
....  | https  | .... |...

I have to select https protocol here, if I starts the https server. similarly, I have to select http protocol, if I starts the http server. Also I don't know what this protocol field stands for.

---

To sum it up:

proxy server | Firefox proxy setting |work? | SwitchOmega setting |work?|
 http        | http + ssl setting    | yes  | protocol http       |yes  |
 https       | http + ssl setting    | no   | protocol https      |yes  |
 https       |      -                |  -   | protocal http       |no   |

---

So my questions are:

1. Can I connect to the https proxy server through the ordinary way(without an extension)? If possible, how?
2. Why can I connect to the https proxy server through SwitchOmega?
3. I think I build a https proxy server. But why others are saying that "There's no such thing as a https proxy server?

---

Source code

https server

var http = require('http');
var https = require('https');
var fs = require('fs');
var net = require('net');
var url = require('url');

console.log("qqqqq2");

function request(cReq, cRes) {
    console.log("request=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("request=====end");
    var u = url.parse(cReq.url);

    var options = {
        hostname : u.hostname, 
        port     : u.port || 80,
        path     : u.path,       
        method     : cReq.method,
        headers     : cReq.headers
    };

    var pReq = http.request(options, function(pRes) {
        cRes.writeHead(pRes.statusCode, pRes.headers);
        pRes.pipe(cRes);
    }).on('error', function(e) {
        cRes.end();
    });

    cReq.pipe(pReq);
    // console.log(cReq.headers);
    // console.log(cReq.method);
    // console.log(cReq.url);
    // console.log("^_^^_^^_^^_^^_^^_^");
    // cRes.writeHead('200');
    // cRes.end('hello world2222\n');
}

function connect(cReq, cSock) {
    console.log("connect=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("connect=====end");
    var u = url.parse('http://' + cReq.url);

    var pSock = net.connect(u.port, u.hostname, function() {
        cSock.write('HTTP/1.1 200 Connection Established\r\n\r\n');
        pSock.pipe(cSock);
    }).on('error', function(e) {
        cSock.end();
    });

    cSock.pipe(pSock);
}

var options = {
    key: fs.readFileSync('./privkey1.pem'),
    cert: fs.readFileSync('./fullchain1.pem')
};

https.createServer(options)
    .on('request', request)
    .on('connect', connect)
    .listen(9999, '0.0.0.0');

http server

var http = require('http');
var net = require('net');
var url = require('url');

console.log('qqqqq2');

function request(cReq, cRes) {
    console.log("request=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("request=====end");

    var u = url.parse(cReq.url);

    var options = {
        hostname : u.hostname, 
        port     : u.port || 80,
        path     : u.path,       
        method     : cReq.method,
        headers     : cReq.headers
    };

    var pReq = http.request(options, function(pRes) {
        cRes.writeHead(pRes.statusCode, pRes.headers);
        pRes.pipe(cRes);
    }).on('error', function(e) {
        cRes.end();
    });

    cReq.pipe(pReq);
}

function connect(cReq, cSock) {
    console.log("connect=====start");
    console.log(cReq.headers);
    console.log(cReq.url);
    console.log(cReq.method);
    console.log("connect=====end");
    var u = url.parse('http://' + cReq.url);

    var pSock = net.connect(u.port, u.hostname, function() {
        cSock.write('HTTP/1.1 200 Connection Established\r\n\r\n');
        pSock.pipe(cSock);
    }).on('error', function(e) {
        cSock.end();
    });

    cSock.pipe(pSock);
}

http.createServer()
    .on('request', request)
    .on('connect', connect)
    .listen(9999, '0.0.0.0');

---

Test Server

You can easily build a http proxy server and test it. But it may be cumbersome to build a https proxy server, because you need to deploy certificates. So a https proxy test server is provided, based on the code above.

Test server is deleted since I've found the answer.

Answer 1

I found the answer in Security StackExchange. Is it possible to connect to a proxy with an ssl (or otherwise encrypted) connection?

From https://wiki.squid-cache.org/Features/HTTPS#Encrypted_browser-Squid_connection :

Encrypted browser-Squid connection

While HTTPS design efforts were focused on end-to-end communication, it would also be nice to be able to encrypt the browser-to-proxy connection (without creating a CONNECT tunnel that blocks Squid from accessing and caching content). This would allow, for example, a secure use of remote proxies located across a possibly hostile network.

Squid can accept regular proxy traffic using https_port in the same way Squid does it using an http_port directive. Unfortunately, popular modern browsers do not permit configuration of TLS/SSL encrypted proxy connections. There are open bug reports against most of those browsers now, waiting for support to appear. If you have any interest, please assist browser teams with getting that to happen.

...

Chrome

The Chrome browser is able to connect to proxies over SSL connections if configured to use one in a PAC file or command line switch. GUI configuration appears not to be possible (yet).

Firefox

The Firefox 33.0 browser is able to connect to proxies over TLS connections if configured to use one in a PAC file. GUI configuration appears not to be possible (yet), though there is a config hack for embedding PAC logic.

More information related to Chrome can be found in http://dev.chromium.org/developers/design-documents/secure-web-proxy.

---

To answer the questions:

1. Can I connect to the https proxy server through the ordinary way(without an extension)? If possible, how?

The traditional way(e.g. Manual proxy configuration field in Firefox) to set a http proxy server is for HTTP proxy server only. One can only set a https proxy via pac files (e.g. Automatic proxy configuration URL field in Firefox).

2. Why can I connect to the https proxy server through SwitchOmega?

The SwitchOmega extension in fact generates a pac file for Chrome to use, though how it interacts with Chrome is so far unknown to me.

By clicking the Export PAC button in SwitchOmega, I get a file contains:

var FindProxyForURL = function(init, profiles) {
    return function(url, host) {
        "use strict";
        var result = init, scheme = url.substr(0, url.indexOf(":"));
        do {
            result = profiles[result];
            if (typeof result === "function") result = result(url, host, scheme);
        } while (typeof result !== "string" || result.charCodeAt(0) === 43);
        return result;
    };
}("+test", {
    "+test": function(url, host, scheme) {
        "use strict";
        if (/^127\.0\.0\.1$/.test(host) || /^::1$/.test(host) || /^localhost$/.test(host)) return "DIRECT";
        return "HTTPS myHttpsProxyServer.com:9999"; // This line matters
    }
});

From https://developer.mozilla.org/en-US/docs/Web/HTTP/Proxy_servers_and_tunneling/Proxy_Auto-Configuration_(PAC)_file:

HTTP host:port   
The specified proxy should be used   
HTTPS host:port 
The specified HTTPS proxy should be used  

3. I think I build a https proxy server. But why others are saying that "There's no such thing as a https proxy server?

Yes I build a https proxy server/a http proxy server over tls connection. Those who says "There's no such thing as a https proxy server" are wrong.