JavaMail: Keeping IMAPFolder.idle() alive

Question

I am making a program that needs to monitor a Gmail account for new messages, and in order to get them ASAP I am using JavaMail's idle feature. Here is a code snippet from the thread I am using to call folder.idle():

//Run method that waits for idle input. If an exception occurs, end the thread's life.
public void run() {

    IMAPFolder folder = null;

            try {
                folder = getFolder();
                while(true)
                {
                  //If connection has been lost, attempt to restore it
                  if (!folder.isOpen())
                      folder = getFolder();
                  //Wait until something happens in inbox
                  folder.idle(true);
                  //Notify controller of event
                  cont.inboxEventOccured();
                }
            }
            catch (Exception ex) {
                ex.printStackTrace();
            }
             System.out.println("MailIdleWaiter thread ending.");
}

The getFolder() method basically opens the connection to the IMAP server and opens the inbox.

This works for a while, but after 10 minutes or so it stops getting updates (no exception is thrown).

I am looking for suggestions to keep the connection alive. Do I need a second thread whose only role is to sleep and renew the idle() thread every 10 minutes or is there an easier/better way?

Thanks in advance.

Answer 1

A common mistake is to assume an IDLE command will keep posting updates indefinitely. However, the RFC 2177, that defines the IDLE extension states:

The server MAY consider a client inactive if it has an IDLE command running, and if such a server has an inactivity timeout it MAY log the client off implicitly at the end of its timeout period. Because of that, clients using IDLE are advised to terminate the IDLE and re-issue it at least every 29 minutes to avoid being logged off. This still allows a client to receive immediate mailbox updates even though it need only "poll" at half hour intervals.

GMail in particular, has a much lower timeout, as you say, around 10 minutes.

We simply need to reissue the IDLE command every 9 minutes or so for it to work. The javax.mail APIs have no way to set a timeout for the IDLE command, so you will need a second thread to move around this.

A first approach would be to have the second thread interrupt the first one, handling the exception and ignoring it. This however, would allow for no clean way to shutdown the thread, so I won't recomend it. A much cleaner way is to have the second thread issue a NOOP command to the server. This does nothing at all, but is enough to have IDLE abort and be reissued.

I here provide some code to do this:

public void startListening(IMAPFolder imapFolder) {
    // We need to create a new thread to keep alive the connection
    Thread t = new Thread(
        new KeepAliveRunnable(imapFolder), "IdleConnectionKeepAlive"
    );

    t.start();
    
    while (!Thread.interrupted()) {
        LOGGER.debug("Starting IDLE");
        try {
            imapFolder.idle();
        } catch (MessagingException e) {
            LOGGER.warn("Messaging exception during IDLE", e);
            throw new RuntimeException(e);
        }
    }
    
    // Shutdown keep alive thread
    if (t.isAlive()) {
        t.interrupt();
    }
}

/**
 * Runnable used to keep alive the connection to the IMAP server
 * 
 * @author Juan Martín Sotuyo Dodero <[email protected]>
 */
private static class KeepAliveRunnable implements Runnable {

    private static final long KEEP_ALIVE_FREQ = 300000; // 5 minutes

    private IMAPFolder folder;

    public KeepAliveRunnable(IMAPFolder folder) {
        this.folder = folder;
    }

    @Override
    public void run() {
        while (!Thread.interrupted()) {
            try {
                Thread.sleep(KEEP_ALIVE_FREQ);
                
                // Perform a NOOP just to keep alive the connection
                LOGGER.debug("Performing a NOOP to keep alvie the connection");
                folder.doCommand(new IMAPFolder.ProtocolCommand() {
                    public Object doCommand(IMAPProtocol p)
                            throws ProtocolException {
                        p.simpleCommand("NOOP", null);
                        return null;
                    }
                });
            } catch (InterruptedException e) {
                // Ignore, just aborting the thread...
            } catch (MessagingException e) {
                // Shouldn't really happen...
                LOGGER.warn("Unexpected exception while keeping alive the IDLE connection", e);
            }
        }
    }
}

Answer 2

Actually the Java Mail samples include an IMAP IDLE example, which is as follows. Besides that, the IdleManager class might be interesting.

/*
 * Copyright (c) 1996-2010 Oracle and/or its affiliates. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 *   - Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *
 *   - Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer in the
 *     documentation and/or other materials provided with the distribution.
 *
 *   - Neither the name of Oracle nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
 * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
 * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
 * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
 * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
 * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
 * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
 * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
 * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
 * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

import java.util.*;
import java.io.*;
import javax.mail.*;
import javax.mail.event.*;
import javax.activation.*;

import com.sun.mail.imap.*;

/* Monitors given mailbox for new mail */

public class monitor {

    public static void main(String argv[]) {
    if (argv.length != 5) {
        System.out.println(
        "Usage: monitor <host> <user> <password> <mbox> <freq>");
        System.exit(1);
    }
    System.out.println("\nTesting monitor\n");

    try {
        Properties props = System.getProperties();

        // Get a Session object
        Session session = Session.getInstance(props, null);
        // session.setDebug(true);

        // Get a Store object
        Store store = session.getStore("imap");

        // Connect
        store.connect(argv[0], argv[1], argv[2]);

        // Open a Folder
        Folder folder = store.getFolder(argv[3]);
        if (folder == null || !folder.exists()) {
        System.out.println("Invalid folder");
        System.exit(1);
        }

        folder.open(Folder.READ_WRITE);

        // Add messageCountListener to listen for new messages
        folder.addMessageCountListener(new MessageCountAdapter() {
        public void messagesAdded(MessageCountEvent ev) {
            Message[] msgs = ev.getMessages();
            System.out.println("Got " + msgs.length + " new messages");

            // Just dump out the new messages
            for (int i = 0; i < msgs.length; i++) {
            try {
                System.out.println("-----");
                System.out.println("Message " +
                msgs[i].getMessageNumber() + ":");
                msgs[i].writeTo(System.out);
            } catch (IOException ioex) { 
                ioex.printStackTrace(); 
            } catch (MessagingException mex) {
                mex.printStackTrace();
            }
            }
        }
        });

        // Check mail once in "freq" MILLIseconds
        int freq = Integer.parseInt(argv[4]);
        boolean supportsIdle = false;
        try {
        if (folder instanceof IMAPFolder) {
            IMAPFolder f = (IMAPFolder)folder;
            f.idle();
            supportsIdle = true;
        }
        } catch (FolderClosedException fex) {
        throw fex;
        } catch (MessagingException mex) {
        supportsIdle = false;
        }
        for (;;) {
        if (supportsIdle && folder instanceof IMAPFolder) {
            IMAPFolder f = (IMAPFolder)folder;
            f.idle();
            System.out.println("IDLE done");
        } else {
            Thread.sleep(freq); // sleep for freq milliseconds

            // This is to force the IMAP server to send us
            // EXISTS notifications. 
            folder.getMessageCount();
        }
        }

    } catch (Exception ex) {
        ex.printStackTrace();
    }
    }
}