How to enable wire logging for a java HttpURLConnection traffic?

Question

I've used Jakarta commons HttpClient in another project and I would like the same wire logging output but using the "standard" HttpUrlConnection.

I've used Fiddler as a proxy but I would like to log the traffic directly from java.

Capturing what goes by the connection input and output streams is not enough because the HTTP headers are written and consumed by the HttpUrlConnection class, so I will not be able to log the headers.

Answer 1

According to Sun's HttpURLConnection source there is some logging support via JUL.

Setup (adjust path as required):

-Djava.util.logging.config.file=/full/path/to/logging.properties 

logging.properties:

handlers= java.util.logging.ConsoleHandler java.util.logging.ConsoleHandler.level = FINEST sun.net.www.protocol.http.HttpURLConnection.level=ALL 

This will log to the console, adjust as required to e.g. log to a file.

Example output:

2010-08-07 00:00:31 sun.net.www.protocol.http.HttpURLConnection writeRequests FIN: sun.net.www.MessageHeader@16caf435 pairs: {GET /howto.html HTTP/1.1: null}{User-Agent: Java/1.6.0_20}{Host: www.rgagnon.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2010-08-07 00:00:31 sun.net.www.protocol.http.HttpURLConnection getInputStream FIN: sun.net.www.MessageHeader@5ac0728 pairs: {null: HTTP/1.1 200 OK}{Date: Sat, 07 Aug 2010 04:00:33 GMT}{Server: Apache}{Accept-Ranges: bytes}{Content-Length: 17912}{Keep-Alive: timeout=5, max=64}{Connection: Keep-Alive}{Content-Type: text/html} 

Note that this prints only headers without body.

See http://www.rgagnon.com/javadetails/java-debug-HttpURLConnection-problem.html for details.

There is also system property -Djavax.net.debug=all. But it's mainly useful for SSL debugging.

Answer 2

I've been able to log all SSL traffic implementing my own SSLSocketFactory on top of the default one.

This worked for me because all of our connections are using HTTPS and we can set the socket factory with the method HttpsURLConnection.setSSLSocketFactory.

A more complete solution that enables monitoring on all sockets can be found at http://www.javaspecialists.eu/archive/Issue169.html Thanks to Lawrence Dol for pointing in the right direction of using Socket.setSocketImplFactory

Here is my not ready for production code:

public class WireLogSSLSocketFactory extends SSLSocketFactory {      private SSLSocketFactory delegate;      public WireLogSSLSocketFactory(SSLSocketFactory sf0) {         this.delegate = sf0;     }      public Socket createSocket(Socket s, String host, int port,             boolean autoClose) throws IOException {         return new WireLogSocket((SSLSocket) delegate.createSocket(s, host, port, autoClose));     }      /*     ...     */      private static class WireLogSocket extends SSLSocket {          private SSLSocket delegate;          public WireLogSocket(SSLSocket s) {             this.delegate = s;         }          public OutputStream getOutputStream() throws IOException {             return new LoggingOutputStream(delegate.getOutputStream());         }          /*         ...         */          private static class LoggingOutputStream extends FilterOutputStream {             private static final Logger logger = Logger.getLogger(WireLogSocket.LoggingOutputStream.class);             //I'm using a fixed charset because my app always uses the same.              private static final String CHARSET = "ISO-8859-1";             private StringBuffer sb = new StringBuffer();              public LoggingOutputStream(OutputStream out) {                 super(out);             }              public void write(byte[] b, int off, int len)                     throws IOException {                 sb.append(new String(b, off, len, CHARSET));                 logger.info("\n" + sb.toString());                 out.write(b, off, len);             }              public void write(int b) throws IOException {                 sb.append(b);                 logger.info("\n" + sb.toString());                 out.write(b);             }              public void close() throws IOException {                 logger.info("\n" + sb.toString());                 super.close();             }         }     } }