Incorrect boundary string in multipart/mixed message

Question

I'm using Delphi 2006 to create and send an email message with an attachment in a personal-use-only application. I send the message with an instance of TIdSMTP, and then also put a copy into a specific IMAP folder with an instance of TIdIMAP4. This all works very nicely with the version of Indy 10 that was distributed with BDS2006, with one exception: the time is always incorrect in the email header.

I decided to fix that if I could, and after searching for a solution it seemed most reasonable to get the latest Indy 10 snapshot and use that.

That puts the correct time into the email header, but there's a new problem. The boundary string is now different in the header of the message that is added to the IMAP folder than what comes in the body of the email! (Please note that the message that was sent via SMTP is correct.)

This is the relevant header information from the older version of Indy 10:

Content-Type: multipart/mixed; boundary="XNlC6OyS4QSiHY2U=_jsXyps6TR34pFNsh"
MIME-Version: 1.0
Date: Tue, 22 Nov 2011 09:11:58 +0000

A test of the BDS2006-bundled version of Indy

--XNlC6OyS4QSiHY2U=_jsXyps6TR34pFNsh
Content-Type: application/octet-stream;
        name="20111122.xls"

And this is the same header information from Indy 10.5.8 (snapshot 10_4702 which I installed yesterday):

Content-Type: multipart/mixed; boundary="CDbEncbFvL7RZdOJ3DOIRoRBs=_nBsbZms"
MIME-Version: 1.0
Date: Tue, 22 Nov 2011 07:33:46 -0600

investigating more deeply, why does the boundary indicator change?

--h=_WzGWJei29fng7SqdPpDh1nkJxJZhiGc
Content-Type: application/octet-stream;
    name="20111122.xls"

The time stamp is fixed, but now the boundary string is incorrect. The result is that there appears to be nothing at all in the message that gets added to my IMAP folder.

Here is the relevant code that creates the email message and attachment, sends it, and puts a copy into the IMAP folder:

  FTheMsg.Date := Now;  // FTheMsg is a component dropped onto the form
  FTheMsg.Recipients.EMailAddresses := edMailTo.Text;
  FTheMsg.ClearBody;
  FTheMsg.Subject := 'Glucose Readings ' + FormatDateTime('mm/dd/yy', FStartDate) + ' - ' +
              FormatDateTime('mm/dd/yy', FEndDate);
  FTheMsg.Body.Assign(FMemo.Lines);

  // create the attachment
  TIdAttachmentFile.Create(FTheMsg.MessageParts, fileName);

  // send the mail!
  FSmtp.Host := FSMTPHost;  // values set up elsewhere, FSmtp is a component on the form
  FImap.Host := FIMAPHost;  // FImap is also a component on the form

  FSmtp.Connect;
  try
    FSmtp.Send(FTheMsg);
    FImap.Connect;
    try
      if (not FImap.AppendMsg('Sent Items', FTheMsg, FTheMsg.LastGeneratedHeaders, [mfSeen]))     then
        StatusBar1.Panels[4].Text := 'Failed append msg';
    finally
      FImap.Disconnect;
    end;
  finally
    FSmtp.Disconnect;
  end;

As I said, the email that gets sent is fine and displays properly. But the one that is added to my IMAP folder (in FImap.AppendMsg() above) is incorrect. I've attempted to trace through the code to see where it might be going wrong, but frankly, I'm not familiar enough with Indy and the various email protocols/RFCs to be able to determine what's going wrong. About all I can tell is that the older version saves the message to a temporary file before appending it to the folder, while the newer version saves it to a memory stream instead. Obviously, something is different about that, but I'm currently too ignorant to determine what.

Is there a simple way to correct the timestamp problem in the old version? If so, that would be fine for my use, as everything else appears to be correct. If not, what else do I need to do to fix the problem exhibited here with the incorrect boundary string?

(As this is an application strictly for my own use, I can live with the incorrect date if I have to, but not with the "empty-appearing" copy in my 'Sent Items' folder.)

If more information is needed, I'll gladly supply whatever I can.

[Edit: I did incorporate something of a kludge in MY code, using the older version of Indy. I simply set the date/time of the message to UTC/GMT time before sending it, and that, at least, allows the message to contain the correct time at the receiver's end. I don't particularly care for that fix, but it does the trick.]

Answer 1

Don't use the TIdMessage.Body property to hold your text when an attachment is present. Put the text into a TIdText object instead. Even better, use the TIdMessageBuilder... classes, such as TIdMessageBuilderPlain, to prepare the TIdMessage body for you.

Try this:

FTheMsg.Clear; 
FTheMsg.Date := Now;  // FTheMsg is a component dropped onto the form 
FTheMsg.Recipients.EMailAddresses := edMailTo.Text; 
FTheMsg.Subject := 'Glucose Readings ' + FormatDateTime('mm/dd/yy', FStartDate) + ' - ' + FormatDateTime('mm/dd/yy', FEndDate); 
FTheMsg.ContentType := 'multipart/mixed'; 

TIdText.Create(FTheMsg.MessageParts, FMemo.Lines).ContentType := 'text/plain';
TIdAttachmentFile.Create(FTheMsg.MessageParts, fileName); 

FSmtp.Connect; 
try 
  FSmtp.Send(FTheMsg); 
  FImap.Connect; 
  try 
    if (not FImap.AppendMsg('Sent Items', FTheMsg, nil, [mfSeen])) then 
      StatusBar1.Panels[4].Text := 'Failed append msg'; 
  finally 
    FImap.Disconnect; 
  end; 
finally 
  FSmtp.Disconnect; 
end; 

Or:

FTheMsg.Clear; 
FTheMsg.Date := Now;  // FTheMsg is a component dropped onto the form 
FTheMsg.Recipients.EMailAddresses := edMailTo.Text; 
FTheMsg.Subject := 'Glucose Readings ' + FormatDateTime('mm/dd/yy', FStartDate) + ' - ' + FormatDateTime('mm/dd/yy', FEndDate); 

with TIdMessageBuilderPlain.Create do
try
  PlainText.Assign(FMemo.Lines);
  Attachments.Add(fileName); 
  FillMessage(FTheMsg);
finally
  Free;
end;

FSmtp.Connect; 
try 
  FSmtp.Send(FTheMsg); 
  FImap.Connect; 
  try 
    if (not FImap.AppendMsg('Sent Items', FTheMsg, nil, [mfSeen])) then 
      StatusBar1.Panels[4].Text := 'Failed append msg'; 
  finally 
    FImap.Disconnect; 
  end; 
finally 
  FSmtp.Disconnect; 
end; 

Now, with that said, it is likely that it will still not work correctly either way. TIdIMAP4.AppendMsg() calls TIdMessage.SaveToStream() internally, which regenerates the email content fresh (and thus alters the boundary used in the body). Whether you pass in the pre-existing TIdMessage.LastGeneratedHeaders or let TIdIMAP4.AppendMsg() grab the current TIdMessage.Headers, they will be out of sync with the new boundary that TIdMessage.SaveToStream() generates.

To make sure both SMTP and IMAP4 are in sync, they need to receive the same data. Try calling TIdMessage.SaveToStream() manually first with the TIdMessage.NoEncode property set to False, then set the TIdMessage.NoDecode property to True and call TIdMessage.LoadFromStream() to reload the saved data as-is into the TIdMessage.Headers and TIdMessage.Body properties, then call TIdSMTP.Send() and TIdIMAP4.AppendMsg() with the TIdMessage.NoEncode property set to True so the TIdMessage.Headers and TIdMessage.Body are sent as-is.

I know, this goes against what the TIdIMAP4.AppendMsg() comments/docs say to do. AppendMsg() does not currently take MIME into account at all, so it does not ensure the MIME boundary in the header and body match each other. I will try to check in a fix for that. For Indy 11, Indy's entire MIME handling system is going to be redesigned, so I'll make sure it is possible to preserve boundaries, and/or to specify custom boundaries, so AppendMsg() can match the body boundary to the header boundary better.

IMAP4 is a very tricky protocol to work with in general.