Collection was modified; enumeration operation may not execute. But why?

Question

All,

I receive "Collection was modified; enumeration operation may not execute." during serialization of my object graph (it fails on SerializeDictionary(),- but I don't have a dictionary in my object graph). I am using Newtonsoft serialize to serialize my object graph.

I know what the exception means and after reviewing my object graph I am thinking there maybe a bug in Newtonsoft json serializer. Below I posted the entire stack trace. Here are the interesting details. I setup another test case to troubleshoot the problem and to my surprise I don't receive the serialization exception. Same object graph, same serializer, just different project environments. Below are the 2 cases, 1 works, the other doesn't work. In both cases I am loading exact object graph from the database!

Case 1: Serialization fails

Visual Studio: 2012 Pro

asp.net 3.5 (CLR 2)

Serializer: Newtonsoft

Project type: Web application

Case 2: Serialization works

Visual Studio: 2005 Pro

asp.net 2.0 (CLR 2)

Serializer: Newtonsoft

Project type: Console app

Serialization Code:

    Receipt rec = ReceiptManager.LoadReceipt(26157);
var result = Newtonsoft.Json.JsonConvert.SerializeObject(rec,
                      new Newtonsoft.Json.JsonSerializerSettings
                      {
                          settings.PreserveReferencesHandling = Newtonsoft.Json.PreserveReferencesHandling.Objects;
                      });

Model (see if you can spot something wrong):

namespace PrlSystems.PimarcCore.DomainModel.Commerce
{
    /// <summary>
    /// Represents a document in accounting.
    /// </summary>
    [Serializable()]
    public class Document
    {
        private string _documentNumber = "";
        private DateTime _documentDate;
        private string _currencyCode;


        public Document()
        {
        }

        public Document(string documentNumber)
        {
            _documentNumber = documentNumber;
        }

        public string DocumentNumber
        {
            get { return _documentNumber; }
            set { _documentNumber = value; }
        }

        public DateTime DocumentDate
        {
            get { return _documentDate; }
            set { _documentDate = value; }
        }

        public string CurrencyCode
        {
            get { return _currencyCode; }
            set { _currencyCode = value; }
        }
    }
}



namespace PrlSystems.PimarcCore.DomainModel.Commerce
{
    [Serializable()]
    public class Receipt : Document
    {
        private int _id;
        private long _versionNumber;

        private ProjectInfo _project;
        private ClientInfo _client;
        private PaymentMethod _paymentMethod;
        private string _referenceNo;
        private string _chequeNo;
        private Money _amountReceived;
        private string _comment;
        private string _templateFile;   // used to render this document (ie. read-only/printing)

        // accounting export related
        private bool _posted;
        private string _accountingId; //  The internal id of the payment as it is in the accounting system.
        private string _accountingBatchId;
        private DateTime? _transactionDate;
        private string _transactionId;
        private int? _transactionNumber;
        private PostingStatus? _postStatus;
        private string _postingUser;    // user name that posted this document

        private ReceiptLineItemCollection _lineItems = new ReceiptLineItemCollection();

        public Receipt()
        {
            _id = -1;
        }

        public Receipt(string currencyCode)
        {
            _id = -1;
            CurrencyCode = currencyCode;
        }

        public int Id
        {
            get { return _id; }
            set { _id = value; }
        }

        public long VersionNumber
        {
            get { return _versionNumber; }
            set { _versionNumber = value; }
        }

        public ProjectInfo Project
        {
            get { return _project; }
            set { _project = value; }
        }

        public ClientInfo Client
        {
            get { return _client; }
            set { _client = value; }
        }

        public PaymentMethod PaymentMethod
        {
            get { return _paymentMethod; }
            set { _paymentMethod = value; }
        }

        public string ReferenceNo
        {
            get { return _referenceNo; }
            set { _referenceNo = value; }
        }

        public string ChequeNo
        {
            get { return _chequeNo; }
            set { _chequeNo = value; }
        }

        public Money AmountReceived
        {
            get { return _amountReceived; }
            set { _amountReceived = value; }
        }

        public string Comment
        {
            get { return _comment; }
            set { _comment = value; }
        }

        public string TemplateFile
        {
            get { return _templateFile; }
            set { _templateFile = value; }
        }



        public bool Posted
        {
            get { return _posted; }
            set { _posted = value; }
        }

        public string AccountingId
        {
            get { return _accountingId; }
            set { _accountingId = value; }
        }

        public string AccountingBatchId
        {
            get { return _accountingBatchId; }
            set { _accountingBatchId = value; }
        }

        public DateTime? TransactionDate
        {
            get { return _transactionDate; }
            set { _transactionDate = value; }
        }

        public string TransactionId
        {
            get { return _transactionId; }
            set { _transactionId = value; }
        }

        public int? TransactionNumber
        {
            get { return _transactionNumber; }
            set { _transactionNumber = value; }
        }

        public PostingStatus? PostStatus
        {
            get { return _postStatus; }
            set { _postStatus = value; }
        }

        public string PostingUser
        {
            get { return _postingUser; }
            set { _postingUser = value; }
        }

        public ReceiptLineItemCollection LineItems
        {
            get { return _lineItems; }
            set { _lineItems = value; }
        }

        public ReceiptLineItem FindReceiptLine(int invoiceId)
        {
            foreach (ReceiptLineItem lineItem in _lineItems)
            {
                if (lineItem.Invoice.Id == invoiceId)
                    return lineItem;
            }
            return null;
        }

    }
}





namespace PrlSystems.PimarcCore.DomainModel.Commerce
{
    public class ReceiptLineItemCollection : List<ReceiptLineItem>
    {
    }
}








namespace PrlSystems.PimarcCore.DomainModel.Commerce
{
    public class ReceiptLineItem
    {
        private InvoiceDetailedSummary _invoice;
        private Money _amountAppliedToInvoice;

        private Receipt _receipt;

        public ReceiptLineItem()
        {
        }

        public ReceiptLineItem(Receipt receipt)
        {
            if (receipt == null)
                throw new ArgumentNullException("receipt");
            _receipt = receipt;
        }

        public InvoiceDetailedSummary Invoice
        { 
            get { return _invoice; }
            set { _invoice = value; } 
        }

        public Money AmountAppliedToInvoice
        { 
            get { return _amountAppliedToInvoice; }
            set { _amountAppliedToInvoice = value; }
        }

        [Obsolete("TODO: To be removed in future release.")]
        public Receipt Receipt
        {
            get { return _receipt; }
            set { _receipt = value; }
        }
    }
}












namespace PrlSystems.PimarcCore.DomainModel.Commerce
{
    /// <summary>
    /// Lightweight object that contains brief invoice summary.
    /// </summary>
    [Serializable()]
    public class InvoiceBriefSummary
    {
        public const string PROP_INVOICEID = "Id";
        public const string PROP_INVOICEDATE = "InvoiceDate";
        public const string PROP_INVOICENUMBER = "InvoiceNumber";
        public const string PROP_CURRENCYCODE = "CurrencyCode";
        public const string PROP_GRANDTOTAL = "GrandTotal";

        private int _id;
        private DateTime _invoiceDate;
        private string _invoiceNumber;
        private string _currencyCode;

        public InvoiceBriefSummary()
        {
        }

        public InvoiceBriefSummary(Invoice invoice)
        {
            if (invoice == null)
                throw new ArgumentNullException("invoice");

            _id = invoice.Id;
            _invoiceDate = invoice.DocumentDate;
            _invoiceNumber = invoice.DocumentNumber;
            _currencyCode = invoice.CurrencyCode;
        }

        public InvoiceBriefSummary(InvoiceHeader invoice)
        {
            if (invoice == null)
                throw new ArgumentNullException("invoice");

            _id = invoice.Id;
            _invoiceDate = invoice.InvoiceDate;
            _invoiceNumber = invoice.InvoiceNumber;
            _currencyCode = invoice.CurrencyCode;            
        }

        public int Id
        {
            get { return _id; }
            set { _id = value; }
        }

        public DateTime InvoiceDate
        {
            get { return _invoiceDate; }
            set { _invoiceDate = value; }
        }

        public string InvoiceNumber
        {
            get { return _invoiceNumber; }
            set { _invoiceNumber = value; }
        }

        public string CurrencyCode
        {
            get { return _currencyCode; }
            set { _currencyCode = value; }
        }
    }
}






namespace PrlSystems.PimarcCore.DomainModel.Commerce
{
    /// <summary>
    /// XML serializer friendly detailed invoice summary.
    /// </summary>
    [Serializable()]
    public class InvoiceDetailedSummary : InvoiceBriefSummary
    {
        private ContactInfo _billTo;
        private ProjectInfo _project;
        private decimal _subtotal;
        private decimal _grandtotal;
        private decimal _taxtotal;

        private string _accountingId;
        private string _transactionId;
        private int? _transactionNumber;

        public InvoiceDetailedSummary()
        {
        }

        public InvoiceDetailedSummary(Invoice invoice)
            : base(invoice)
        {
            _billTo = invoice.BillToAddress;
            _project = invoice.Project;
            _subtotal = invoice.Subtotal.Amount;

            TaxCalculator calculator = new TaxCalculator();
            TaxSummary tax = calculator.GetDocumentTax(invoice);
            _taxtotal = tax.Taxes.GetTotalTax().Amount;
            _grandtotal = invoice.GetGrandtotal(tax).Amount;

            _accountingId = invoice.AccountingId;
            _transactionId = invoice.TransactionId;
            _transactionNumber = invoice.TransactionNumber;
        }

        public InvoiceDetailedSummary(InvoiceHeader invoice)
            : base(invoice)
        {
            _billTo = invoice.BillTo.GetContactInformation();
            _project = invoice.Project.GetProjectInfo();
            _project.ClientId = invoice.Client.Id; // use client ID on the invoice
            _subtotal = invoice.Subtotal;
            _grandtotal = invoice.Grandtotal;

            _accountingId = invoice.AccountingId;
            _transactionId = invoice.TransactionId;
            _transactionNumber = invoice.TransactionNumber;
        }

        public ContactInfo BillTo
        {
            get { return _billTo; }
            set { _billTo = value; }
        }

        public ProjectInfo Project
        {
            get { return _project; }
            set { _project = value; }
        }

        public decimal Subtotal
        {
            get { return _subtotal; }
            set { _subtotal = value; }
        }

        public decimal Taxtotal
        {
            get { return _taxtotal; }
            set { _taxtotal = value; }
        }

        public decimal Grandtotal
        {
            get { return _grandtotal; }
            set { _grandtotal = value; }
        }


        public string AccountingId
        {
            get { return _accountingId; }
            set { _accountingId = value; }
        }

        public string TransactionId
        {
            get { return _transactionId; }
            set { _transactionId = value; }
        }

        public int? TransactionNumber
        {
            get { return _transactionNumber; }
            set { _transactionNumber = value; }
        }
    }
}

Complete stack trace:

    System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
 at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
 at System.Collections.Generic.Dictionary`2.Enumerator.MoveNext()
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeDictionary(JsonWriter writer, IDictionary values, JsonDictionaryContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeISerializable(JsonWriter writer, ISerializable value, JsonISerializableContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeISerializable(JsonWriter writer, ISerializable value, JsonISerializableContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeISerializable(JsonWriter writer, ISerializable value, JsonISerializableContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
 at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
 at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
 at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings)
 at Pimarcnet.UserControls.Invoicing.ReceiptForm.get_ViewModel() in c:\PRL\DEV\Visual Studio 2012\Pimarcnet35\Pimarcnet35\UserControls\Invoicing\ReceiptForm.ascx.cs:line 42
 at ASP.usercontrols_invoicing_receiptform_ascx.__Render__control1(HtmlTextWriter __w, Control parameterContainer) in c:\PRL\DEV\Visual Studio 2012\Pimarcnet35\Pimarcnet35\UserControls\Invoicing\ReceiptForm.ascx:line 164
 at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
 at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.Control.Render(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
 at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.Control.Render(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
 at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.HtmlControls.HtmlForm.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)
 at System.Web.UI.HtmlControls.HtmlForm.Render(HtmlTextWriter output)
 at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.HtmlControls.HtmlForm.RenderControl(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
 at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.HtmlControls.HtmlContainerControl.Render(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
 at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.Control.Render(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children)
 at System.Web.UI.Control.RenderChildren(HtmlTextWriter writer)
 at System.Web.UI.Page.Render(HtmlTextWriter writer)
 at System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter)
 at System.Web.UI.Control.RenderControl(HtmlTextWriter writer)
 at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
 --- End of inner exception stack trace ---
 at System.Web.UI.Page.HandleError(Exception e)
 at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
 at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
 at System.Web.UI.Page.ProcessRequest()
 at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
 at System.Web.UI.Page.ProcessRequest(HttpContext context)
 at ASP.pages_invoicing_editreceipt2_aspx.ProcessRequest(HttpContext context) in c:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\root\07d8779e\a6e74bf1\App_Web_hhnohwoa.4.cs:line 0
 at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
 at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) 

Answer 1

All, I found the culprit that was causing this exception,- and it wasn't easy. Posting for others to benefit.

On the Receipt class, the PaymentMethod property (specifically the type) was causing the serializer to blow up. When the receipt object was hydrated (from database entities), the referenced PaymentMethod property was actually pointing to a database entity that was loaded lazily (ie. referencing a hibernate proxy). As the serializer was serializing the object graph, hibernate was injecting the real entity causing the error,- at least that's what I think.

It still doesn't explain the following (I don't have time to investigate further since I need the project to be done):

1) In my other environment this was not an issue.

2) As for kicks, I loaded (in a non-lazy way) a list of PaymentMethod objects and tried to serialize that ... this triggered the exception again. PaymentMethod class has few simple properties and it doesn't reference any other possibly lazy loaded entities. Interestingly, I tried loading a list of Accounts (another type that has for most part identical fields) and serializing that. And this worked! Really strange.

The moral of the story:

If you see "Collection was modified; enumeration operation may not execute", look at your object graph and see if you are referencing an object that maybe loaded lazily. If you still can't figure out the problem, simplify your object graph, ie. instead of having complex objects nested, extract the fields from the nested onto the root object and the serialize the root. This is exactly how I found my problem.