Jetty WebSocket proxying

Question

Just wonder if anyone has experimented with WebSocket proxying (for transparent proxy) using embedded Jetty?

After about a day and a half playing with Jetty 9.1.2.v20140210, all I can tell is that it can't proxy WebSockets in its current form, and adding such support is non-trivial task (afaict at least).

Basically, Jetty ProxyServlet strips out the "Upgrade" and "Connection" header fields regardless of whether it's from a WebSocket handshake request. Adding these fields back is easy as shown below. But, when the proxied server returned a response with HTTP code 101 (switching protocols), no protocol upgrade is done on the proxy server. So, when the first WebSocket packet arrives, the HttpParser chokes and see that as a bad HTTP request.

If anyone already has a solution for it or is familiar with Jetty to suggest what to try, that would be very much appreciated.

Below is the code in my experiment stripping out the unimportant bits:

    public class ProxyServer
    {
        public static void main(String[] args) throws Exception
        {
            Server server = new Server();
            ServerConnector connector = new ServerConnector(server);
            connector.setPort(8888);
            server.addConnector(connector);

            // Setup proxy handler to handle CONNECT methods
            ConnectHandler proxy = new ConnectHandler();
            server.setHandler(proxy);

            // Setup proxy servlet
            ServletContextHandler context = new ServletContextHandler(proxy, "/", ServletContextHandler.SESSIONS);
            ServletHolder proxyServlet = new ServletHolder(MyProxyServlet.class);
            context.addServlet(proxyServlet, "/*");

            server.start();
        }
    }

    @SuppressWarnings("serial")
    public class MyProxyServlet extends ProxyServlet
    {
        @Override
        protected void customizeProxyRequest(Request proxyRequest, HttpServletRequest request)
        {
            // Pass through the upgrade and connection header fields for websocket handshake request. 
            String upgradeValue = request.getHeader("Upgrade");
            if (upgradeValue != null && upgradeValue.compareToIgnoreCase("websocket") == 0)
            {
                setHeader(proxyRequest, "Upgrade", upgradeValue);
                setHeader(proxyRequest, "Connection", request.getHeader("Connection"));
            }
        }

        @Override
        protected void onResponseHeaders(HttpServletRequest request, HttpServletResponse response, Response proxyResponse)
        {
            super.onResponseHeaders(request, response, proxyResponse);

            // Restore the upgrade and connection header fields for websocket handshake request.
            HttpFields fields = proxyResponse.getHeaders();
            for (HttpField field : fields)
            {
                if (field.getName().compareToIgnoreCase("Upgrade") == 0)
                {
                    String upgradeValue = field.getValue();
                    if (upgradeValue != null && upgradeValue.compareToIgnoreCase("websocket") == 0)
                    {
                        response.setHeader(field.getName(), upgradeValue);
                        for (HttpField searchField : fields)
                        {
                            if (searchField.getName().compareToIgnoreCase("Connection") == 0) {
                                response.setHeader(searchField.getName(), searchField.getValue());
                            }
                        }
                    }
                }
            }
        }
    }

Answer 1

Let's imagine the proxy scheme that you are trying to build, we have client A, server B and proxy P. Now let's walk through connect workflow:

1. A established TCP connection with proxy P (A-P)
2. A sends CONNECT addr(B) request with WebSocket handshake

Here you have the first problem, by HTTP RFC headers used in WS handshake are not end-to-end headers, because for HTTP they make sense only on a transport layer (between two hops).

3. P establishes TCP connection to B (P-B)
4. P sends WS handshake HTTP request to B
5. B responds with HTTP->WS upgrade (by sending 101)

And here is another problem, after sending HTTP 101 server B and client A now will communicate only over TCP, but jetty servlet doesn't support plain TCP packets propagation. In other words jetty proxy servlet waits till client A will start transmitting HTTP request, which will never happen after A will receive HTTP 101.

You would need to implement this by yourself using WS server and WS client.