IMAP - How to search for all messages in a conversation thread?

Question

I'm working on an IMAP client, and would like to be able to find a list of all messages that are referenced in a conversation thread.

I know that the "References" header includes a list of messages referenced in a conversation, so I tried searching it like so:

TAG1 UID SEARCH all HEADER References "<CAOZnC-Nr+Q+bS_Nn5XiZ6A-kw=ZRBYrNbdoRfgZ65OjGA4_BKg@mail.gmail.com>"

But it returns nothing. I've successfully searched for a single message using the "Message-ID" header, like so:

TAG2 UID SEARCH all HEADER Message-ID "<[email protected]>"

Is there anyway to do this using IMAP 4?

NOTE: I'm aware that searches only work on 1 mailbox at a time, but at least half of these messages are in the target folder for my searches, and they don't show up in my search results.

Answer 1

Your search query is wrong -- you should remove the ALL token from there -- what you're sending is not syntactically valid. A correct form is this one, for example:

1 UID SEARCH HEADER references "<CAOZnC-Nr+Q+bS_Nn5XiZ6A-kw=ZRBYrNbdoRfgZ65OjGA4_BKg@mail.gmail.com>"

That shall get you going.

However, please keep in mind that the References header of a nested message might not contain everything what a message earlier in the thread will contain -- the generic line length limits apply, and this means that the total size of the header is limited and some of the "middle" items might get removed.

There are methods to overcome this limitation, like the INTHREAD operator defined in an experimental extension which is supported by Dovecot (and no other IMAP servers, AFAIK). Using that, you can simply ask for all UIDs within a thread where a particular message is present, like this:

1 UID SEARCH INTHREAD REFS HEADER Message-Id "something"

I've actually tested this with Dovecot and it works (and has worked for years). Please be advised that Dovecot does not support the MESSAGEID search key, though, and that the syntax for INTHREAD REFS is different than what the draft standard says. The command above works, though.

If you need to work without any extensions, then it looks like you have no other chance but to either:

• Fetch HEADER.FIELDS[Message-Id References In-Reply-To] at once and analyze the messages client-side,
• Whenever your SEARCH HEADER References returns a new UID, fetch HEADER.FIELDS[References In-Reply-To], extract the "new" message-ids from there, add them at the end of your queue and proceed further.

Finally, you could also rely on GMail's X-GM-THRID if you target mainly Gmail users, but please advised that their implementation is rather limited, including a hard limit of at most 100 messages per thread, non-standard thread correlation etc etc.

Well, pick your poison.