java SSLEngine says NEED_WRAP, call .wrap() and still NEED_WRAP

Question

I am seeing a weird issue with SSLEngine and wondering if there is an issue with my code or SSLEngine. Here is the order in which I see things

1. HandshakeStatus is NEED_WRAP
2. We call SSLEngine.WRAP
3. after, there is ZERO data written to the buffer, and SSLEngineResult.result=OK(not overflow nor underflow :( ) and HandshakeStatus is STILL NEED_WRAP

Most important question: How to debug thoroughly? How to 'see' each message somehow? I can capture the byte stream easily enough but is there some library that can parse that into SSL handshake objects?

line 298 (recording previous handshake status) to line 328(where we throw the exception with info) is the relevant code here

https://github.com/deanhiller/webpieces/blob/sslEngineFartingExample/core/core-ssl/src/main/java/org/webpieces/ssl/impl/AsyncSSLEngine3Impl.java

The stack trace was

2019-06-21 08:58:24,562 [-] [webpiecesThreadPool6] Caller+1  at org.webpieces.util.threading.SessionExecutorImpl$RunnableWithKey.run(SessionExecutorImpl.java:123)
  ERROR: Uncaught Exception
java.lang.IllegalStateException: Engine issue.  hsStatus=NEED_WRAP status=OK previous hsStatus=NEED_WRAP
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.sendHandshakeMessageImpl(AsyncSSLEngine3Impl.java:328)
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.sendHandshakeMessage(AsyncSSLEngine3Impl.java:286)
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.doHandshakeWork(AsyncSSLEngine3Impl.java:133)
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.doHandshakeLoop(AsyncSSLEngine3Impl.java:246)
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.unwrapPacket(AsyncSSLEngine3Impl.java:210)
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.doWork(AsyncSSLEngine3Impl.java:109)
    at org.webpieces.ssl.impl.AsyncSSLEngine3Impl.feedEncryptedPacket(AsyncSSLEngine3Impl.java:82)
    at org.webpieces.nio.impl.ssl.SslTCPChannel$SocketDataListener.incomingData(SslTCPChannel.java:175)
    at org.webpieces.nio.impl.threading.ThreadDataListener$1.run(ThreadDataListener.java:26)
    at org.webpieces.util.threading.SessionExecutorImpl$RunnableWithKey.run(SessionExecutorImpl.java:121)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

any ideas? How can I really dig into this further? My preference is a library that takes bytes and spits out ssl objects representing each handshake message or decrypted packet(with any header info that comes with the original encrypted thing).

Specifically, here is the code mentioned above

        HandshakeStatus previousStatus = sslEngine.getHandshakeStatus();
    //CLOSE and all the threads that call feedPlainPacket can have contention on wrapping to encrypt and
    //must synchronize on sslEngine.wrap
    Status lastStatus;
    HandshakeStatus hsStatus;
    synchronized (wrapLock ) {

        HandshakeStatus beforeWrapHandshakeStatus = sslEngine.getHandshakeStatus();
        if (beforeWrapHandshakeStatus != HandshakeStatus.NEED_WRAP)
            throw new IllegalStateException("we should only be calling this method when hsStatus=NEED_WRAP.  hsStatus=" + beforeWrapHandshakeStatus);

        //KEEEEEP This very small.  wrap and then listener.packetEncrypted
        SSLEngineResult result = sslEngine.wrap(SslMementoImpl.EMPTY, engineToSocketData);
        lastStatus = result.getStatus();
        hsStatus = result.getHandshakeStatus();
    }

    log.trace(()->mem+"write packet pos="+engineToSocketData.position()+" lim="+
                    engineToSocketData.limit()+" status="+lastStatus+" hs="+hsStatus);

    if(lastStatus == Status.BUFFER_OVERFLOW || lastStatus == Status.BUFFER_UNDERFLOW)
        throw new RuntimeException("status not right, status="+lastStatus+" even though we sized the buffer to consume all?");

    boolean readNoData = engineToSocketData.position() == 0;
    engineToSocketData.flip();
    try {
        CompletableFuture<Void> sentMsgFuture;
        if(readNoData) {
            log.trace(() -> "ssl engine is farting. READ 0 data.  hsStatus="+hsStatus+" status="+lastStatus);

            throw new IllegalStateException("Engine issue.  hsStatus="+hsStatus+" status="+lastStatus+" previous hsStatus="+previousStatus);
            //A big hack since the Engine was not working in live testing with FireFox and it would tell us to wrap
            //and NOT output any data AND not BufferOverflow.....you have to do 1 or the other, right!
            //instead cut out of looping since there seems to be no data
            //sslEngineIsFarting = true;
            //sentMsgFuture = CompletableFuture.completedFuture(null);

thanks, Dean

Answer 1

System.setProperty("jdk.tls.server.protocols", "TLSv1.2");

System.setProperty("jdk.tls.client.protocols", "TLSv1.2");

The system property should be set before the JSSE get loaded. For example, set the property within command line. Is it the cause that the system property does not work for you?

... the SSLEngine tells us we need to WRAP which is correct as we need to be replying with a close_notify as well to prevent truncation attacks BUT instead the engine returns 0 bytes and tells us the state of the engine is still NEED_WRAP.

TLS 1.3 is using a half-close policy (See RFC 8446). When receiving the close_notify, the inbound side will be closed and the outbound side keeps open. The local side cannot receive any data, but is allowed to send more application data.

There are a few compatibility impact by the half-close policy (See JDK 11 release note, https://www.oracle.com/technetwork/java/javase/11-relnote-issues-5012449.html). The system property, "jdk.tls.acknowledgeCloseNotify", can be used as a workaround. For more details, please refer to the JDK 11 release note.

Answer 2

oh, even better, downgrading to 1.8.0_111 yields success

2019-06-30 00:11:54,813 [main] Caller+1  at 
WEBPIECESxPACKAGE.DevelopmentServer.main(DevelopmentServer.java:32)
 INFO: Starting Development Server under java version=1.8.0_111

webpiecesThreadPool5, READ: TLSv1.2 Alert, length = 26
webpiecesThreadPool2, RECV TLSv1.2 ALERT:  warning, close_notify
webpiecesThreadPool2, closeInboundInternal()
webpiecesThreadPool2, closeOutboundInternal()
webpiecesThreadPool5, RECV TLSv1.2 ALERT:  warning, close_notify
webpiecesThreadPool5, closeInboundInternal()
webpiecesThreadPool5, closeOutboundInternal()
webpiecesThreadPool2, SEND TLSv1.2 ALERT:  warning, description = close_notify
webpiecesThreadPool5, SEND TLSv1.2 ALERT:  warning, description = close_notify
webpiecesThreadPool2, WRITE: TLSv1.2 Alert, length = 26
webpiecesThreadPool5, WRITE: TLSv1.2 Alert, length = 26