Why is android HttpURLConnection not sending back FIN?

Question

From my Android app I wanted to Post data to server and get the response back, process it then send back and get another request. Since it is continuous communication until no more response to process, I prefer to go with HttpURLConnection with http.keepAlive = true.

And my attempt to reuse socket is successful, but the problems I am facing are:

1. I am trying to initiate Close from Client (Android App), since if the termination starts from Server then Server goes to TIME_WAIT state. And I don't want my server to go to that state so I preferred my Client to initiate Termination. But unfortunately I find no suitable way to do it with HttpURLConnection
2. After hours of searching I gave up on doing the above attempt and went with initiating Close from Server based on keepalivetimeout, but when Server sends FIN, Client responds with just ACK, because of that Connection held up on FIN_WAIT_2 in server and CLOSE_WAIT in agent.

Source Code:

private HttpStatus communicateWithServer(String httpUrl, String dataToSend, boolean keepAlive) {
    HttpStatus status = new HttpStatus(HTTP_STATUS_FAILURE);
    
    try {
        
        initializeConnection(httpUrl,keepAlive);
        postDataToConnection(connection, dataToSend);
        status = readDataFromConnection(connection);
        
    } catch (MalformedURLException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
        readErrorStreamAndPrint(connection);
    }
    connection.disconnect();
    return status;
}

/**
 * API to close connection, calling this will not force the connection to shutdown
 * this will work based on the Connection header set.
 * @param connection
 */
public void closeConnection(){
    if(connection != null){
        connection.disconnect();
    }
}


/**
 * Used to initialize the HttpURLConnection for the given url
 * All properties required for connection are preset here 
 * Connection Time Out : 20 Seconds
 * Connection Type     : keep alive
 * Content Type        : application/json;charset=UTF-8
 * And also All certificates will be evaluated as Valid.[ TODO will be removed soon]
 * @param httpUrl
 * @return
 * @throws MalformedURLException
 * @throws IOException
 */
private void initializeConnection(String httpUrl, boolean keepAlive) throws MalformedURLException, IOException{
    
    URL url = new URL(httpUrl);
    connection = (HttpsURLConnection) url.openConnection();
    
    connection.setConnectTimeout(20000);
    connection.setReadTimeout(20000);
    connection.setDoInput(true);
    connection.setDoOutput(true);
    connection.setRequestMethod("POST");                                                                        //NO I18N
    connection.setRequestProperty("Content-Type", "application/json;charset=UTF-8");                            //NO I18N
    connection.setRequestProperty("Connection", "Keep-Alive");      //NO I18N
}


/**
 * API to post data to given connection 
 * call to this API will close the @OutputStream
 * @param connection
 * @param data
 * @throws IOException
 */
private void postDataToConnection(URLConnection connection , String data) throws IOException{
    
    OutputStream outStream = connection.getOutputStream();
    BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(outStream));
    writer.write(data);
    writer.flush();
    writer.close();
    outStream.close();
}

/**
 * API to read error stream and log
 * @param connection
 */
private void readErrorStreamAndPrint(URLConnection connection){
    try{
        InputStream inStream = ((HttpURLConnection) connection).getErrorStream();
        String responseData = "";
        String line;
        BufferedReader br=new BufferedReader(new InputStreamReader(inStream));
        while ((line=br.readLine()) != null) {
            responseData+=line;
        }
        
    } catch (IOException ioe) {
        
    }
}


/**
 * API to read data from given connection and return 
 * call to this API will close the @InputStream
 * @param connection
 * @return
 * @throws IOException
 */
private HttpStatus readDataFromConnection(URLConnection connection) throws IOException{
    
    HttpStatus status = new HttpStatus(HTTP_STATUS_FAILURE);
    int responseCode=((HttpURLConnection) connection).getResponseCode();
    InputStream inStream = connection.getInputStream();
    String responseData = "";
    if (responseCode == HttpURLConnection.HTTP_OK) {
        responseData = readStreamAsString(inStream);
        status.setStatus(HTTP_STATUS_SUCCESS);
        status.setUrlDataBuffer(responseData);
    }
    else {
        status.setStatus(HTTP_STATUS_FAILURE);
    }
    inStream.close();
    return status;
}

/**
 * Read the InputStream to String until EOF
 * Call to this API will not close @InputStream
 * @param inStream
 * @return
 * @throws IOException
 */
private String readStreamAsString(InputStream inStream) throws IOException{
    StringBuilder responseData = new StringBuilder();
    String line;
    BufferedReader br=new BufferedReader(new InputStreamReader(inStream));
    while ((line=br.readLine()) != null) {
        responseData.append(line);
    }
    return responseData.toString();
}

Can anyone help?

Answer 1

When you use http.keepAlive = true connections start to be recycled, in a connection pool, and are kept open. Even if the server closes the connection it's still listening and the client still thinks it can send data. After all, the server's FIN only says the server will not send more data.

Since the internal logic of the connection pool is inaccessible you have little control. Nevertheless, as you use https, you have a open window to the lower layers and can gain access to the created Socket through HttpsURLConnection.setSSLSocketFactory(SSLSocketFactory).

You can create a wrapper for the default factory (SSLSocketFactory.getDefault()) that allows to close the Socket. Something like the simplification bellow:

public class MySSLSocketFactory extends SSLSocketFactory {

    private SSLSocketFactory factory = (SSLSocketFactory) SSLSocketFactory.getDefault();
    private Socket last;

    public void closeLastSocket() {
        if (last != null) {
            last.close();
        }
    }

    public Socket createSocket() throws IOException {
        return this.last = factory.createSocket();
    }

    ...

}

When you close the underlying socket then the connection should not be eligible for recycling and will be discarded. So if you do closeLastSocket and disconnect the connection should not even go to the pool and if you do the other way around the connection will should be discarded only when you create a new connection.