EWS reports SendItem as succeeding but message is still in Drafts folder

Question

I'm using the EWS Managed API to send emails via exchange. When sending an item with attachments I first do a CreateItem and then a SendItem. Mostly this works fine but sometimes items are left in the Drafts folder, even though EWS reports the SendItem succeeds. How can I tell what's going on here?

The EWS messages I see from tracing are first CreateItem:

  <?xml version="1.0" encoding="utf-8"?>
  <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Header>
      <t:RequestServerVersion Version="Exchange2013" />
    </soap:Header>
    <soap:Body>
      <m:CreateItem MessageDisposition="SaveOnly">
        <m:SavedItemFolderId>
          <t:DistinguishedFolderId Id="drafts">
            <t:Mailbox>
              <t:EmailAddress>[email protected]</t:EmailAddress>
            </t:Mailbox>
          </t:DistinguishedFolderId>
        </m:SavedItemFolderId>

        ...

      </m:CreateItem>
    </soap:Body>
  </soap:Envelope>
</Trace>

I get a Success response:

  <?xml version="1.0" encoding="utf-8"?>
  <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
      <h:ServerVersionInfo MajorVersion="15" MinorVersion="0" MajorBuildNumber="1104" MinorBuildNumber="3" Version="V2_22" xmlns:h="http://schemas.microsoft.com/exchange/services/2006/types" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" />
    </s:Header>
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
      <m:CreateItemResponse xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
        <m:ResponseMessages>
          <m:CreateItemResponseMessage ResponseClass="Success">
            <m:ResponseCode>NoError</m:ResponseCode>
            <m:Items>
              <t:Message>
                <t:ItemId Id="AAMkADExMmY0MTgzLWJmZmUtNDcxNi1iOTk0LWMzZTU3M2I3NjBkMwBGAAAAAABL/dBAdDByRbztYn12Cy/4BwCJZGY3HQ5zQpbF9WoRXD0sAAAAAAEPAACJZGY3HQ5zQpbF9WoRXD0sAAA6HNVuAAA=" ChangeKey="CQAAABYAAACJZGY3HQ5zQpbF9WoRXD0sAAA6JC4z" />
                <t:Attachments>
                  <t:FileAttachment>
                    <t:AttachmentId Id="AAMkADExMmY0MTgzLWJmZmUtNDcxNi1iOTk0LWMzZTU3M2I3NjBkMwBGAAAAAABL/dBAdDByRbztYn12Cy/4BwCJZGY3HQ5zQpbF9WoRXD0sAAAAAAEPAACJZGY3HQ5zQpbF9WoRXD0sAAA6HNVuAAABEgAQAGiKX5lgtuFLgfIBgyg4IwM=" />
                  </t:FileAttachment>
                  <t:FileAttachment>
                    <t:AttachmentId Id="AAMkADExMmY0MTgzLWJmZmUtNDcxNi1iOTk0LWMzZTU3M2I3NjBkMwBGAAAAAABL/dBAdDByRbztYn12Cy/4BwCJZGY3HQ5zQpbF9WoRXD0sAAAAAAEPAACJZGY3HQ5zQpbF9WoRXD0sAAA6HNVuAAABEgAQADkNd4D6H0NAgxCM7uH6MGo=" />
                  </t:FileAttachment>
                  <t:FileAttachment>
                    <t:AttachmentId Id="AAMkADExMmY0MTgzLWJmZmUtNDcxNi1iOTk0LWMzZTU3M2I3NjBkMwBGAAAAAABL/dBAdDByRbztYn12Cy/4BwCJZGY3HQ5zQpbF9WoRXD0sAAAAAAEPAACJZGY3HQ5zQpbF9WoRXD0sAAA6HNVuAAABEgAQAKu2RdYFlBVIiLyPxshNCZQ=" />
                  </t:FileAttachment>
                  <t:FileAttachment>
                    <t:AttachmentId Id="AAMkADExMmY0MTgzLWJmZmUtNDcxNi1iOTk0LWMzZTU3M2I3NjBkMwBGAAAAAABL/dBAdDByRbztYn12Cy/4BwCJZGY3HQ5zQpbF9WoRXD0sAAAAAAEPAACJZGY3HQ5zQpbF9WoRXD0sAAA6HNVuAAABEgAQAJ2C2tyz2iJHm8XC9LmSkcA=" />
                  </t:FileAttachment>
                </t:Attachments>
              </t:Message>
            </m:Items>
          </m:CreateItemResponseMessage>
        </m:ResponseMessages>
      </m:CreateItemResponse>
    </s:Body>
  </s:Envelope>
</Trace>

I then ask to send it and move to SentItems:

  <?xml version="1.0" encoding="utf-8"?>
  <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Header>
      <t:RequestServerVersion Version="Exchange2013" />
    </soap:Header>
    <soap:Body>
      <m:SendItem SaveItemToFolder="true">
        <m:ItemIds>
          <t:ItemId Id="AAMkADExMmY0MTgzLWJmZmUtNDcxNi1iOTk0LWMzZTU3M2I3NjBkMwBGAAAAAABL/dBAdDByRbztYn12Cy/4BwCJZGY3HQ5zQpbF9WoRXD0sAAAAAAEPAACJZGY3HQ5zQpbF9WoRXD0sAAA6HNVuAAA=" ChangeKey="CQAAABYAAACJZGY3HQ5zQpbF9WoRXD0sAAA6JC4z" />
        </m:ItemIds>
        <m:SavedItemFolderId>
          <t:DistinguishedFolderId Id="sentitems">
            <t:Mailbox>
              <t:EmailAddress>[email protected]</t:EmailAddress>
            </t:Mailbox>
          </t:DistinguishedFolderId>
        </m:SavedItemFolderId>
      </m:SendItem>
    </soap:Body>
  </soap:Envelope>
</Trace>

And I get a Success NoError response:

  <?xml version="1.0" encoding="utf-8"?>
  <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
      <h:ServerVersionInfo MajorVersion="15" MinorVersion="0" MajorBuildNumber="1104" MinorBuildNumber="3" Version="V2_22" xmlns:h="http://schemas.microsoft.com/exchange/services/2006/types" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" />
    </s:Header>
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
      <m:SendItemResponse xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
        <m:ResponseMessages>
          <m:SendItemResponseMessage ResponseClass="Success">
            <m:ResponseCode>NoError</m:ResponseCode>
          </m:SendItemResponseMessage>
        </m:ResponseMessages>
      </m:SendItemResponse>
    </s:Body>
  </s:Envelope>
</Trace>

These are all traced by the same managed ThreadId so I'm pretty sure they all correspond to the same item.

I'm not using exchange impersonation. I'm authenticating with username & password auth with credentials for the mailbox that is sending the items.

EDIT (22 May):

I've added X-AnchorMailbox header to my requests and upgraded to EWS Managed API 2.2 (was using 2.1, although since the SOAP messages looked right I don't think this was an issue). The problem is still happening.

I added code to re-retrieve the item after sending, to check if it's still in Drafts folder. I do this through EmailMessage.Bind() with the Id of the item just sent from drafts. When I run against Exchange on Office 365 that always raises an exception because the item no longer exists, which is expected since the ItemId changes when the item moves folder. When I run against customer's Exchange server it finds the item by that Id. I then try to resend it and get a ServiceResponseException with message Access is denied. Check credentials and try again. If I try to resend a message against Office 365 I get a SendItemResponse with message The specified object was not found in the store., The process failed to get the correct properties. (SOAP message is SendItemResponse with ResponseCode ErrorItemNotFound.

I've raised a support request with Microsoft (which oddly hasn't been responded to in over 48 hours ... maybe there's some trick to raising support requests).

EDIT (24 May)

When repeating the tests I sometimes saw items that remain in Drafts BUT are also received by the recipient! When inspecting their Properties > Internet Headers via outlook they both have the same Message-ID. More indication that something is messed up on the Exchange side of things?

EDIT (30 May)

I heard from Microsoft support who explained a bit about how sending in exchange works and what they think is happening:

a No error response from EWS means that the message was successfully flagged for submissions (PR_MESSAGE_FLAGS > MSGFLAG_SUBMIT). When the submission flag is set, the Mail Submission service on Exchange notifies the Transport Service that there are messages awaiting to be picked up in the Drafts (Outbox if Outlook). If any client reads the message after it’s been flagged for submission but before it’s picked up by the Transport service, the MSGFLAG_SUBMIT flag is removed which leads to the message being stuck in the Drafts folder.

  In order to determine if this is the case, you can sign up for notifications using Streaming Notifications (and a tool the likes of EWSEditor or EwsStreaming) and check for any item Modified events and check for any PR_MESSAGE_FLAGS changes.

  As discussed, the message could have been accessed by any client or API before the transport service went to pick it up, such as OWA, Outlook, EWS Managed API, Outlook add-ins or any mobile apps.

So next task is twofold:

1. Identify if anything is reading my messages immediately after they've flagged to be sent, resetting the MSGFLAG_SUBMIT flag. If so, ask them to please stop.

2. Include a process to re-send items that I think have been sent that I later find in Drafts.

I didn't get any information about why I'd get an Access Denied when trying to resend, but perhaps that's consistent with something else modifying the item.

They didn't think there was any chance this is a problem in EWS, where EWS is incorrectly reporting success when actually the message hasn't been flagged to be sent. Which is reassuring.

EDIT (20 June):

Introducing a process to re-send items that are stuck in Drafts didn't work very well. It turns out that sometimes when items are left in Drafts they've actually been sent and received by the recipient, so re-sending them means the recipient gets a duplicate of the email. Or several, if the same problem happens again. Sometimes the item isn't sent but is still removed from Drafts and moved to Sent Items - again it's not possible for the sender to work out whether it's sent or not. Totally weird. I've asked another question here about specifically whether Exchange message sending is expected to be atomic and reliable.

---

Also posed on Exchange forums

Answer 1

Taken from MSDN documentation on SendItem:

An additional scenario to consider is when a delegate creates an e-mail message and saves it to the Drafts folder of the delegate's mailbox. If the delegate tries to send the item and save a copy to the principal's Sent Items distinguished folder, the message is sent correctly, the draft message remains in the delegate's Drafts folder, the sent message does not appear in either the delegate's or principal's Sent Items folder, and the response is a success.

The documentation seems to say you're out of luck... The best work around may be to perform the send and move to sent items as two separate functions as well.