Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to establish a happens-before relationship between a request handling thread and a SocketChannel selector thread?

Consider a request-response protocol.

We spawn a thread to perform a select() loop for reads and writes on an accepted non-blocking SocketChannel. That might look something like

while (!isStopped()) {
    selector.select();
    Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();

    while (selectedKeys.hasNext()) {
        SelectionKey selectedKey = selectedKeys.next();
        selectedKeys.remove();
        Context context = (Context) selectedKey.attachment();
        if (selectedKey.isReadable()) {
            context.readRequest();
        } else /* if (selectedKey.isWritable()) */ {
            context.writeResponse();
        }
    }
}

where Context is just a container for the corresponding SocketChannel, a buffer and logic to read into it and write from it. The readRequest might look like

public void readRequest() {
    // read all content
    socketChannel.read(requestBuffer);
    // not interested anymore
    selectionKey.interestOps(0);
    executorService.submit(() -> {
        // handle request with request buffer and prepare response
        responseBuffer.put(/* some response content */); // or set fields of some bean that will be serialized

        // notify selector, ready to write
        selectionKey.interestOps(SelectionKey.OP_WRITE);
        selectionKey.selector().wakeup(); // worried about this
    });
}

In other words, we read from the socket channel, populate some buffer and hand off the handling to some other thread. That thread does the handling and prepares a response which it stores in a response buffer. It then notifies the selector that it wants to write and wakes it up.

The Javadoc for Selector#wakeup() doesn't mention any happens-before relationship so I'm worried the selector thread might see the response buffer (or some intermediate object) in an inconsistent state.

Is that a possible scenario? If it is, what's the correct way to hand off a response to be written to a SocketChannel by a Selector loop thread? (Publishing the response through some volatile field? Using a SelectionKey attachment? Some other form of synchronization?)

like image 651
Savior Avatar asked Oct 07 '17 14:10

Savior


1 Answers

Firstly, you don't need to notify the selector about wanting to write. You just write. Only in the case where the write returned zero does the selector or its thread need to be involved.

Secondly, the happens-before relationship occurs as a result of the selector's three levels of synchronization, provided you also do a synchronization, as below.

Your code may block in the interestOps() call if the selector is currently selecting. The possibility is not excluded by the Javadoc. You need to do the operations in the right order:

  1. Wakeup.
  2. Synchronize on the selector.
  3. Call interestOps().

The combination of (2) and the selector's own internal synchronizations establishes any necessary happens-before relationships.

like image 166
user207421 Avatar answered Oct 13 '22 00:10

user207421