Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Serializing an object using Json.Net causes Out of Memory exception

Disclaimer: I did went through most of the solution provided here but most of them were talking about OOM exception while Deserialization.

I am trying to serialize an object( it's a Tree) into Json using Json.Net. Everything works fine for small objects but i get OOM exception when i try it with large objects. As it works with smaller object of same datatype i am assuming there is no circular reference (I did inspect my data structure for it). Is there a way where i can convert my object into stream ( this is a Windows Store app ) and generate the Json using that stream ?

 public static async Task<bool> SerializeIntoJson<T>(string fileName, StorageFolder destinationFolder, Content content)
    {
        ITraceWriter traceWriter = new MemoryTraceWriter();
        try
        {

            string jsonString = JsonConvert.SerializeObject(content, Formatting.Indented, new JsonSerializerSettings
            {
                PreserveReferencesHandling = PreserveReferencesHandling.Objects,
                TypeNameHandling = TypeNameHandling.All,
                Error = ReportJsonErrors,
                TraceWriter = traceWriter,
                StringEscapeHandling = StringEscapeHandling.EscapeNonAscii
            });
            System.Diagnostics.Debug.WriteLine(traceWriter);

            StorageFile file = await destinationFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
            await Windows.Storage.FileIO.WriteTextAsync(file, jsonString);
            return true;
        }
        catch (NullReferenceException nullException)
        {
            System.Diagnostics.Debug.WriteLine(traceWriter);
            logger.LogError("Exception happened while serializing input object, Error: " + nullException.Message);
            return false;
        }
        catch (Exception e)
        {
            System.Diagnostics.Debug.WriteLine(traceWriter);
            logger.LogError("Exception happened while serializing input object, Error: " + e.Message, e.ToString());
            return false;
        }
    }

In order to convert my object into stream, the code i found out was using a BinaryFormatter which is not available in Windows store app dll's.

like image 666
Jack_2060 Avatar asked Sep 30 '14 19:09

Jack_2060


People also ask

What does serializing a JSON mean?

JSON is a format that encodes objects in a string. Serialization means to convert an object into that string, and deserialization is its inverse operation (convert string -> object).

What is a JSON serialization exception?

JsonSerializationException(String, Exception) Initializes a new instance of the JsonSerializationException class with a specified error message and a reference to the inner exception that is the cause of this exception.

Can JSON be serialized?

JSON is a format that encodes an object to a string. On the transmission of data or storing is a file, data need to be in byte strings, but as complex objects are in JSON format. Serialization converts these objects into byte strings which is JSON serialization.


2 Answers

Updated Code based on suggestions in the comments on the question, This works!

public static async Task<bool> SerializeIntoJson<T>(string fileName, StorageFolder destinationFolder, Content content)
    {
        try
        {
            StorageFile file = await destinationFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
            using (var stream = await file.OpenStreamForWriteAsync())
            {

                StreamWriter writer = new StreamWriter(stream);
                JsonTextWriter jsonWriter = new JsonTextWriter(writer);
                JsonSerializer ser = new JsonSerializer();

                ser.Formatting = Newtonsoft.Json.Formatting.Indented;
                ser.PreserveReferencesHandling = PreserveReferencesHandling.Objects;
                ser.TypeNameHandling = TypeNameHandling.All;
                ser.Error += ReportJsonErrors;

                ser.Serialize(jsonWriter, content);

                jsonWriter.Flush();

            }
            return true;
        }
        catch (NullReferenceException nullException)
        {

            logger.LogError("Exception happened while serializing input object, Error: " + nullException.Message);
            return false;
        }
        catch (Exception e)
        {

            logger.LogError("Exception happened while serializing input object, Error: " + e.Message, e.ToString());
            return false;
        }
    }
like image 187
rene Avatar answered Sep 20 '22 03:09

rene


It is due to very large number of records you are trying to serialize, which occupies large memory. Solutions which I have found for this error as directly writing to the documents using StreamWriter(JsonWriter or TextWriter).

If you have Object use TextWrite

using (TextWriter textWriter = File.CreateText("LocalJsonFile.json"))
{
    var serializer = new JsonSerializer();
    serializer.Serialize(textWriter , yourObject);
}

If you have string use StringWriter

  StringBuilder sb = new StringBuilder();
  StringWriter sw = new StringWriter(sb);

  using(JsonWriter textWriter = new JsonTextWriter(sw))
  {
     var serializer = new JsonSerializer();
     serializer.Serialize(textWriter, yourObject);
  }
like image 26
Dilip0165 Avatar answered Sep 18 '22 03:09

Dilip0165