Java https proxy (using https.proxyPort and https.proxyHost)

Question

I was making a Java application that relies on setting http.proxyPort and http.proxyHost. There are two processes: One is the regular program, the other is the proxy. I have a simple socket listener running on http.proxyPort (which I control). It's as simple as

while (true) {
    try {
    Socket connection = server.accept();

    Handler handler = new Handler(connection);
    handler.start();
    } catch (Exception ex) {
        ex.printStackTrace();
    }
}

So whenever "process 1" makes an http request - like

URL yahoo = new URL("http://www.google.ca/");
URLConnection yc = yahoo.openConnection();
System.out.println(yc.getClass().getName());
BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));

It goes through the proxy. Now what if the client is using an HTTPS protocol? Like instead use https://google.ca? There's a property https.proxyPort and https.proxyHost, but I've literally been trying for months (on and off, it's not too important) without luck. I've read a bunch of threads (I will list some at the end so you know I have done something).

My closest attempt so far: Server

try {
    System.setProperty("javax.net.ssl.keyStore", "test.jks");
    System.setProperty("javax.net.ssl.keyStorePassword", "2520xe");

    SSLServerSocketFactory sslserversocketfactory =
            (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
    SSLServerSocket sslserversocket =
            (SSLServerSocket) sslserversocketfactory.createServerSocket(9999);
    System.out.println("Ready");
    SSLSocket sslsocket = (SSLSocket) sslserversocket.accept();

    InputStream inputstream = sslsocket.getInputStream();
    InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
    BufferedReader bufferedreader = new BufferedReader(inputstreamreader);

    OutputStream toClient = sslsocket.getOutputStream();
    toClient.write(("HTTP/1.0 200 Connection established\n" +
            "Content-Length: " + "Shut down!".getBytes().length
                                     + "\r\n").getBytes("utf-8"));
    toClient.write("Shut down!".getBytes("utf-8"));
    toClient.close();
} catch (Exception exception) {
    exception.printStackTrace();
}

Client

try {
    System.setProperty("https.proxyHost", "127.0.0.1");
    System.setProperty("https.proxyPort", "9999");
    URL yahoo = new URL("https://www.google.ca/");
    URLConnection yc = yahoo.openConnection();
    System.out.println(yc.getClass().getName());
    BufferedReader in = new BufferedReader(
                new InputStreamReader(
                yc.getInputStream()));
    String inputLine;

    while ((inputLine = in.readLine()) != null) 
    System.out.println(inputLine);
    in.close();
} catch (Exception ex) {
    ex.printStackTrace();
}

And I get this error javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection? I googled it but came up with some mail stuff instead.

Basically, I need to create a java proxy server, that's set to the client by the https.proxyPort and https.proxyHost flags, and can send data back to the client app, which may not be modified in any way (it's just using URL connection = new URL("https://..."))

A few of the sites I tried...

• creating a Java Proxy Server that accepts HTTPS
• http://stilius.net/java/java_ssl.php
• There was something else about getting Java to accept all certificates, but I can't find any of the links. I have the code, but I encountered more errors than the thing I'm doing right now, but I can include it if it helps (I didn't initially because this is already a long question)

Answer 1

As auntyellow commented: you don't need to do any SSL-fiddling yourself. Basically https-proxying is about forwarding binary data between two parties.

To cite draft-luotonen-web-proxy-tunneling-01.txt:

 CLIENT -> SERVER                        SERVER -> CLIENT
 --------------------------------------  -----------------------------------
 CONNECT home.netscape.com:443 HTTP/1.0
 User-agent: Mozilla/4.0
 <<< empty line >>>
                                         HTTP/1.0 200 Connection established
                                         Proxy-agent: Netscape-Proxy/1.1
                                         <<< empty line >>>
              <<< data tunneling to both directions begins >>>

So basically you need to ensure you trust your client enough to connect from your proxies firewall-position to the given host and port. Because of this common practice is to limit allowed port to 443, reject connection to localhost and from "untrusted" parties.

This is a "simple" server which is usable as https.proxy in Java if you are not jet convinced:

import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * Created for http://stackoverflow.com/q/16351413/1266906.
 */
public class Server extends Thread {

    public static void main(String[] args) {
        (new Server()).run();
    }

    public Server() {
        super("Server Thread");
    }

    @Override
    public void run() {
        try (ServerSocket serverSocket = new ServerSocket(9999)) {
            Socket socket;
            try {
                while ((socket = serverSocket.accept()) != null) {
                    (new Handler(socket)).start();
                }
            } catch (IOException e) {
                e.printStackTrace();  // TODO: implement catch
            }
        } catch (IOException e) {
            e.printStackTrace();  // TODO: implement catch
            return;
        }
    }

    public static class Handler extends Thread {
        public static final Pattern CONNECT_PATTERN = Pattern.compile("CONNECT (.+):(.+) HTTP/(1\\.[01])",
                                                                      Pattern.CASE_INSENSITIVE);
        private final Socket clientSocket;
        private boolean previousWasR = false;

        public Handler(Socket clientSocket) {
            this.clientSocket = clientSocket;
        }

        @Override
        public void run() {
            try {
                String request = readLine(clientSocket);
                System.out.println(request);
                Matcher matcher = CONNECT_PATTERN.matcher(request);
                if (matcher.matches()) {
                    String header;
                    do {
                        header = readLine(clientSocket);
                    } while (!"".equals(header));
                    OutputStreamWriter outputStreamWriter = new OutputStreamWriter(clientSocket.getOutputStream(),
                                                                                   "ISO-8859-1");

                    final Socket forwardSocket;
                    try {
                        forwardSocket = new Socket(matcher.group(1), Integer.parseInt(matcher.group(2)));
                        System.out.println(forwardSocket);
                    } catch (IOException | NumberFormatException e) {
                        e.printStackTrace();  // TODO: implement catch
                        outputStreamWriter.write("HTTP/" + matcher.group(3) + " 502 Bad Gateway\r\n");
                        outputStreamWriter.write("Proxy-agent: Simple/0.1\r\n");
                        outputStreamWriter.write("\r\n");
                        outputStreamWriter.flush();
                        return;
                    }
                    try {
                        outputStreamWriter.write("HTTP/" + matcher.group(3) + " 200 Connection established\r\n");
                        outputStreamWriter.write("Proxy-agent: Simple/0.1\r\n");
                        outputStreamWriter.write("\r\n");
                        outputStreamWriter.flush();

                        Thread remoteToClient = new Thread() {
                            @Override
                            public void run() {
                                forwardData(forwardSocket, clientSocket);
                            }
                        };
                        remoteToClient.start();
                        try {
                            if (previousWasR) {
                                int read = clientSocket.getInputStream().read();
                                if (read != -1) {
                                    if (read != '\n') {
                                        forwardSocket.getOutputStream().write(read);
                                    }
                                    forwardData(clientSocket, forwardSocket);
                                } else {
                                    if (!forwardSocket.isOutputShutdown()) {
                                        forwardSocket.shutdownOutput();
                                    }
                                    if (!clientSocket.isInputShutdown()) {
                                        clientSocket.shutdownInput();
                                    }
                                }
                            } else {
                                forwardData(clientSocket, forwardSocket);
                            }
                        } finally {
                            try {
                                remoteToClient.join();
                            } catch (InterruptedException e) {
                                e.printStackTrace();  // TODO: implement catch
                            }
                        }
                    } finally {
                        forwardSocket.close();
                    }
                }
            } catch (IOException e) {
                e.printStackTrace();  // TODO: implement catch
            } finally {
                try {
                    clientSocket.close();
                } catch (IOException e) {
                    e.printStackTrace();  // TODO: implement catch
                }
            }
        }

        private static void forwardData(Socket inputSocket, Socket outputSocket) {
            try {
                InputStream inputStream = inputSocket.getInputStream();
                try {
                    OutputStream outputStream = outputSocket.getOutputStream();
                    try {
                        byte[] buffer = new byte[4096];
                        int read;
                        do {
                            read = inputStream.read(buffer);
                            if (read > 0) {
                                outputStream.write(buffer, 0, read);
                                if (inputStream.available() < 1) {
                                    outputStream.flush();
                                }
                            }
                        } while (read >= 0);
                    } finally {
                        if (!outputSocket.isOutputShutdown()) {
                            outputSocket.shutdownOutput();
                        }
                    }
                } finally {
                    if (!inputSocket.isInputShutdown()) {
                        inputSocket.shutdownInput();
                    }
                }
            } catch (IOException e) {
                e.printStackTrace();  // TODO: implement catch
            }
        }

        private String readLine(Socket socket) throws IOException {
            ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
            int next;
            readerLoop:
            while ((next = socket.getInputStream().read()) != -1) {
                if (previousWasR && next == '\n') {
                    previousWasR = false;
                    continue;
                }
                previousWasR = false;
                switch (next) {
                    case '\r':
                        previousWasR = true;
                        break readerLoop;
                    case '\n':
                        break readerLoop;
                    default:
                        byteArrayOutputStream.write(next);
                        break;
                }
            }
            return byteArrayOutputStream.toString("ISO-8859-1");
        }
    }
}