Serializing an array of objects one at a time with JSON.NET

Question

I try to serialize multiple items to json, and formats them as an array.

It's event based, cause it's lot of data which could not be hold in memory as a whole collection. But I need to serialize each item into a file and format it like an array.

_jsonWriter = new JsonTextWriter(new StreamWriter("Output.json")));
DataGatherer.ItemGathered += item =>
{
    _jsonSerializer.Serialize(_jsonWriter, item);
    _jsonWriter.Flush();
};

Currently this outputs following:

{
  "Id": 218515,
  "Name": "A"
}{
  "Id": 118647,
  "Name": "B"
}

Since the serializer serializes each item as an object an does not know its an array.

So how can I tell the JSON.Net serializer to handle each item as an item of an array and formats the data like so:

[{ "Id": 218515, "Name": "A"},{"Id": 118647,"Name": "B"}]

Thanks for any hints!

Answer 1

I would create a simple JsonItemWriter class to wrap the JsonTextWriter. The class would just need to keep track of whether any items had been written to the output yet. If not, use the inner JsonTextWriter to write a StartArray to the stream before writing the item. When the outer ItemWriter is closed, write an EndArray to the stream. Then change your event handler to use the ItemWriter instead of the JsonTextWriter.

Here is what I had in mind for the JsonItemWriter:

class JsonItemWriter
{
    private JsonTextWriter innerWriter;
    private JsonSerializer serializer;

    public JsonItemWriter(JsonTextWriter innerWriter, JsonSerializer serializer)
    {
        this.innerWriter = innerWriter;
        this.serializer = serializer;
    }

    public void WriteItem(object item)
    {
        if (innerWriter.WriteState == Newtonsoft.Json.WriteState.Start)
        {
            innerWriter.WriteStartArray();
        }
        serializer.Serialize(innerWriter, item);
        innerWriter.Flush();
    }

    public void Close()
    {
        innerWriter.WriteEndArray();
        innerWriter.Close();
    }
}

Then set up your event handler like this:

_jsonWriter = new JsonTextWriter(new StreamWriter("Output.json"));
_itemWriter = new JsonItemWriter(_jsonWriter, _jsonSerializer);

DataGatherer.ItemGathered += item =>
{
    _itemWriter.WriteItem(item);
};

Here's a short demo using a mock DataGatherer to stand in for yours:

class Program
{
    static void Main(string[] args)
    {
        JsonSerializer jsonSerializer = new JsonSerializer();
        JsonTextWriter jsonWriter = new JsonTextWriter(new StreamWriter("Output.json"));
        JsonItemWriter itemWriter = new JsonItemWriter(jsonWriter, jsonSerializer);

        MockDataGatherer gatherer = new MockDataGatherer();
        gatherer.ItemGathered += item =>
        {
            itemWriter.WriteItem(item);
        };

        var items = new[]
        {
            new { Id = 218515, Name = "A" },
            new { Id = 118647, Name = "B" }
        };
        gatherer.SimulateReceivingItems(items);

        itemWriter.Close();

        using (StreamReader reader = new StreamReader("Output.json"))
        {
            Console.WriteLine(reader.ReadToEnd());
        }
    }
}

class MockDataGatherer
{
    public void SimulateReceivingItems(IEnumerable<object> items)
    {
        foreach (object item in items)
        {
            ItemGathered(item);
        }
    }

    public event ItemGatheredEventHandler ItemGathered;
    public delegate void ItemGatheredEventHandler(object item);
}

Here is the output of the above (notice the objects are now wrapped in an array):

[{"Id":218515,"Name":"A"},{"Id":118647,"Name":"B"}]

Answer 2

Why don't you build an IEnumerable<YourObject>. It does not need to loead all the collection in your memory and when you give it to the serialiazer, it is parsed as an array. A simple way to do that would be to replace your ItemGathered event call by a simple yield return As you