JavaMail search by ReceivedDate, doesn't work down to the second?

Question

In IMAP, message numbering is based on when a message is placed into a folder (i.e. first message in the folder is 1, second message in is 2, etc.). However, this ordering has no regard for a message's received date. What I'm concerned about are messages older than message number 1 of the folder, i.e. a message that's been moved into a folder by a user manually (and not by when it was received into the system).

I cannot just get every message's receivedDate, as this is horrendoulsy inefficent.

I thought I could do a JavaMail search to get all messages with a received date older than the first message's received date, (and likewise then do a search to get all message with a recieved date newer than the first message's received date... which would be many, but while I need to process ALL of the older ones, I only need to process a few of the newer ones, (and by process, I mean download its headers).

However, when I am using the search, it just doesn't seem to work right.

        Message[] messages = csqFolder.getMessages();           

        if (messages != null && messages.length > 0) {

            Date receivedDate = messages[0].getReceivedDate();
            log.trace("Message 1 receivedDate: <" + receivedDate.toString() + ">");
            SearchTerm msgsOlderThanOETFirst = 
                new ReceivedDateTerm(DateTerm.LT, receivedDate);

            SearchTerm msgsNewerThanFirst = 
                new ReceivedDateTerm(DateTerm.GT, receivedDate);

            Message[] oldMsgs = csqFolder.search(msgsOlderThanOETFirst, messages);
            log.trace("Size of oldMsgs: <" + oldMsgs.length + ">");
            Message[] newMsgs = csqFolder.search(msgsNewerThanFirst, messages);
            log.trace("Size of newMsgs: <" + newMsgs.length + ">");

However, when I run these searches, it does not seem to give the correct results.

On a folder with 8 messages, 7 of which have a received date time of around 12:00 pm on Aug 5, but the first message in the folder which is at 4:00 pm on Aug 5, we will see the following:

Message 1 receivedDate: <Fri Aug 05 16:46:57 CDT 2011>
Size of oldMsgs: <0>
Size of newMsgs: <7>

However, ALL of the remaining seven messages are older than the first message... they should ALL be in oldMsgs.

That said, IF I put a message from the previous day (Aug 4) in that folder, then the search will coreectly work... for that ONE message. It's like the search only works on a day-by-day basis, not down to the second...

I should note that in the term msgsOlderThanOETFirst, I originally was using LE (hence the name), however, that would flip the above resutls -- ALL the messages would then be found in oldMsgs.

Can anyone else confirm if this behavior is true, and if so, would it point to a bug in Java?

I tried to look into the source code of search but I think I was getting a command line version of it and not the kind that the javamail package uses....

I'm using MS Exchange 2007 with SP1, and JavaMail 1.4.3.

Thanks for any suggestions.

UPDATE: I don't think my reading of the comparator order is wrong. Look the answer given in the following thread: java imap fetch messages since a date

Suppose we use that answer, and we have our boundaries from 7/15/2011 to 9/15/2011. Then we have the search compare a message with a given date of 8/4/2011. Then we would have the following:

                                              8/4/2011  <     9/15/2011
SearchTerm olderThen = new ReceivedDateTerm(ComparisonTerm.LT, someFutureDate);
SearchTerm newerThen = new ReceivedDateTerm(ComparisonTerm.GT, somePastDate);
                                              8/4/2011  >     7/15/2011

This would evaluate to true in both cases, which is what we expect, since the date is in the desired range.

Likewise, my code is as follows:

    SearchTerm msgsOlderThanOETFirst =    8/4/2011  < 8/5/2011
            new ReceivedDateTerm(DateTerm.LT, receivedDate); --> TRUE
    SearchTerm msgsNewerThanFirst =       8/4/2011  > 8/5/2011
            new ReceivedDateTerm(DateTerm.GT, receivedDate); --> FALSE

The above evaluations to TRUE and FALSE are what I am expecting, and will receive. However, If we take a given message at 8/5/2011 12:00:00, we get the following:

    SearchTerm msgsOlderThanOETFirst =    8/5/2011 12:00:00  < 8/5/2011 16:46:00
            new ReceivedDateTerm(DateTerm.LT, receivedDate); --> TRUE?
    SearchTerm msgsNewerThanFirst =       8/4/2011 12:00:00  > 8/5/2011 16:46:00
            new ReceivedDateTerm(DateTerm.GT, receivedDate); --> FALSE?

Except, no -- we don't get that -- instead, we get the reverse..... it's easy to think yourself in circles at this point, but... I've double-checked this several times. Is there something wrong with JavaMail, or am I utterly confused? Please correct my confustion if it is the latter!

Answer 1

According to JM developer Bill Shannon, this is a limitation of the IMAP protocol:

from    Bill Shannon 
to  Jonathan Hanson 
cc  [email protected]
date    Wed, Aug 10, 2011 at 11:55 AM
subject Re: Bug with searching by ReceivedDateTerms
mailed-by   oracle.com
    Important mainly because of your interaction with messages in the conversation.

hide details 11:55 AM (16 minutes ago)

The searching is being done by the IMAP server, and you're running into a limitation of the IMAP protocol.