Newtonsoft.Json.JsonSerializationException (Error getting value from 'Value' on 'System.Data.SqlTypes.SqlDouble) serializing SqlGeography

Question

I tried to serialize a DataTable object to Json using Newtonsoft.Json version 'Newtonsoft.Json.10.0.3' in a database SQL server 2012.

The table has a column with type 'geography', which contains instances of type SqlGeography.

The code used to generate json:

    public string SerializeToJson()
    {

     var connstring1 ="Data Source=server1;Initial Catalog=database1;user=xxx;password=yyy";
        var sql = "SELECT  * FROM table_1 "; //table_1 has a column of type geography
        using (var c1 = new SqlConnection(connstring1))
        {
            c1.Open();
            var da = new SqlDataAdapter()
            {
                SelectCommand = new SqlCommand(sql, c1)
            };

            DataSet ds1 = new DataSet("table");
            da.Fill(ds1, "table");
            var dt = ds1.Tables[0];

            //serialize to Json

            try
            {
                var options = new JsonSerializerSettings
                {
                    Formatting = Formatting.None
                };
                //this line fire exception for geography type
                var json = JsonConvert.SerializeObject(dt, options);
                return json;
            }
            catch (Exception ex)
            {

                Console.WriteLine(ex);
            }                
        }
    }

I already installed the assembly 'Microsoft.SqlServer.Types' from feature Pack of sql 2012

I have created a complete C# program (independent of sql server installation) using a datatable with SqlGeography column to show the problem Try it

I get the error:

Newtonsoft.Json.JsonSerializationException: Error getting value from 'Value' on 'System.Data.SqlTypes.SqlDouble'. --->

System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values. at System.Data.SqlTypes.SqlDouble.get_Value() at GetValue(Object ) at Newtonsoft.Json.Serialization.DynamicValueProvider.GetValue(Object target)

I reached https://github.com/JamesNK/Newtonsoft.Json/issues/993, but it can't help.

Any help to resolve the problem.

Edit:

Based on @dbc comments, I provided the complete source code used to generate json.

The complete error message is:

Newtonsoft.Json.JsonSerializationException: Error getting value from 'Value' on 'System.Data.SqlTypes.SqlDouble'. ---> >System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values. at System.Data.SqlTypes.SqlDouble.get_Value() at GetValue(Object ) at Newtonsoft.Json.Serialization.DynamicValueProvider.GetValue(Object target)

--- End of inner exception stack trace --- at Newtonsoft.Json.Serialization.DynamicValueProvider.GetValue(Object target)

at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue( JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member , JsonContainerContract containerContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject (JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue( JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member , JsonContainerContract containerContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.Serialization.JsonSerializerProxy.SerializeInternal(JsonWriter jsonWriter, Object value, Type rootType) at Newtonsoft.Json.Converters.DataTableConverter.WriteJson(JsonWriter writer, Object value, JsonSerializer serializer) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConver table(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty)

at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue( JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member , JsonContainerContract containerContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings) at JsonTester.SerializeToJson() in F:\JsonTester.cs:line 104

Edit2:

I enable tracing as described by @dbc, and get the following log:

2017-08-24T19:45:31.941 Info Started serializing System.Data.DataTable with converter Newtonsoft.Json.Converters.DataTableConverter. Path ''.

2017-08-24T19:45:31.972 Info Started serializing Microsoft.SqlServer.Types.SqlGeography. Path '[0].f1'.

2017-08-24T19:45:32.003 Info Started serializing System.Data.SqlTypes.SqlInt32.Path '[0].f1.STSrid'.

2017-08-24T19:45:32.003 Info Finished serializing System.Data.SqlTypes.SqlInt32. Path '[0].f1.STSrid'.

2017-08-24T19:45:32.003 Info Started serializing System.Data.SqlTypes.SqlDouble. Path '[0].f1.Lat'.

2017-08-24T19:45:32.003 Info Finished serializing System.Data.SqlTypes.SqlDouble. Path '[0].f1.Lat'.

2017-08-24T19:45:32.003 Info Started serializing System.Data.SqlTypes.SqlDouble. Path '[0].f1.Long'.

2017-08-24T19:45:32.003 Info Finished serializing System.Data.SqlTypes.SqlDouble. Path '[0].f1.Long'.

2017-08-24T19:45:32.003 Info Started serializing System.Data.SqlTypes.SqlDouble. Path '[0].f1.Z'.

2017-08-24T19:45:32.003 Error Error serializing System.Data.SqlTypes.SqlDouble.Error getting value from 'Value' on 'System.Data.SqlTypes.SqlDouble'.

2017-08-24T19:45:32.003 Error Error serializing System.Data.DataTable. Error getting value from 'Value' on 'System.Data.SqlTypes.SqlDouble'.

Answer 1

It looks as though the primitive types such as SqlDouble from System.Data.SqlTypes cannot be serialized out-of-the-box by Json.NET because they do not implement their own TypeConverter. From the docs:

Primitive Types

.Net: TypeConverter (convertible to String)
JSON: String

This it will be necessary to implement a custom JsonConverter to serialize these types. Json.NET has several built-in converters such as KeyValuePairConverter for built-in .Net types so this is not unusual.

The fact that SqlBoolean, SqlBinary, SqlDouble and so on do not share a common base class or interface other than INullable requires some duplicated-looking code:

public static class SqlPrimitiveConverters
{
    public static JsonSerializerSettings AddSqlConverters(this JsonSerializerSettings settings)
    {
        foreach (var converter in converters)
            settings.Converters.Add(converter);
        return settings;
    }

    static readonly JsonConverter[] converters = new JsonConverter[]
    {
        new SqlBinaryConverter(),
        new SqlBooleanConverter(),
        new SqlByteConverter(),
        new SqlDateTimeConverter(),
        new SqlDecimalConverter(),
        new SqlDoubleConverter(),
        new SqlGuidConverter(),
        new SqlInt16Converter(),
        new SqlInt32Converter(),
        new SqlInt64Converter(),
        new SqlMoneyConverter(),
        new SqlSingleConverter(),
        new SqlStringConverter(),
        // TODO: converters for primitives from System.Data.SqlTypes that are classes not structs:
        // SqlBytes, SqlChars, SqlXml
        // Maybe SqlFileStream
    };
}

abstract class SqlPrimitiveConverterBase<T> : JsonConverter where T : struct, INullable, IComparable
{
    protected abstract object GetValue(T sqlValue);

    public override bool CanConvert(Type objectType)
    {
        return typeof(T) == objectType;
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        T sqlValue = (T)value;
        if (sqlValue.IsNull)
            writer.WriteNull();
        else
        {
            serializer.Serialize(writer, GetValue(sqlValue));
        }
    }
}

class SqlBinaryConverter : SqlPrimitiveConverterBase<SqlBinary>
{
    protected override object GetValue(SqlBinary sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlBinary.Null;
        return (SqlBinary)serializer.Deserialize<byte[]>(reader);
    }
}

class SqlBooleanConverter : SqlPrimitiveConverterBase<SqlBoolean>
{
    protected override object GetValue(SqlBoolean sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlBoolean.Null;
        return (SqlBoolean)serializer.Deserialize<bool>(reader);
    }
}

class SqlByteConverter : SqlPrimitiveConverterBase<SqlByte>
{
    protected override object GetValue(SqlByte sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlByte.Null;
        return (SqlByte)serializer.Deserialize<byte>(reader);
    }
}

class SqlDateTimeConverter : SqlPrimitiveConverterBase<SqlDateTime>
{
    protected override object GetValue(SqlDateTime sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlDateTime.Null;
        return (SqlDateTime)serializer.Deserialize<DateTime>(reader);
    }
}

class SqlDecimalConverter : SqlPrimitiveConverterBase<SqlDecimal>
{
    protected override object GetValue(SqlDecimal sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlDecimal.Null;
        return (SqlDecimal)serializer.Deserialize<decimal>(reader);
    }
}

class SqlDoubleConverter : SqlPrimitiveConverterBase<SqlDouble>
{
    protected override object GetValue(SqlDouble sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlDouble.Null;
        return (SqlDouble)serializer.Deserialize<double>(reader);
    }
}

class SqlGuidConverter : SqlPrimitiveConverterBase<SqlGuid>
{
    protected override object GetValue(SqlGuid sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlGuid.Null;
        return (SqlGuid)serializer.Deserialize<Guid>(reader);
    }
}

class SqlInt16Converter : SqlPrimitiveConverterBase<SqlInt16>
{
    protected override object GetValue(SqlInt16 sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlInt16.Null;
        return (SqlInt16)serializer.Deserialize<short>(reader);
    }
}

class SqlInt32Converter : SqlPrimitiveConverterBase<SqlInt32>
{
    protected override object GetValue(SqlInt32 sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlInt32.Null;
        return (SqlInt32)serializer.Deserialize<int>(reader);
    }
}

class SqlInt64Converter : SqlPrimitiveConverterBase<SqlInt64>
{
    protected override object GetValue(SqlInt64 sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlInt64.Null;
        return (SqlInt64)serializer.Deserialize<long>(reader);
    }
}

class SqlMoneyConverter : SqlPrimitiveConverterBase<SqlMoney>
{
    protected override object GetValue(SqlMoney sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlMoney.Null;
        return (SqlMoney)serializer.Deserialize<decimal>(reader);
    }
}

class SqlSingleConverter : SqlPrimitiveConverterBase<SqlSingle>
{
    protected override object GetValue(SqlSingle sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlSingle.Null;
        return (SqlSingle)serializer.Deserialize<float>(reader);
    }
}

class SqlStringConverter : SqlPrimitiveConverterBase<SqlString>
{
    protected override object GetValue(SqlString sqlValue) { return sqlValue.Value; }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return SqlString.Null;
        return (SqlString)serializer.Deserialize<string>(reader);
    }
}

Working .Net fiddle forked from your own.

If you need to deserialize the JSON thereby created, you have two additional problems. Firstly, Some of the properties of SqlGeography such as Lat and Long are get-only. You will need to create a custom JsonConverter to fully deserialize this type.

Secondly, Json.NET does not have the ability to deserialize JSON with complex objects for row values to an untyped DataTable. Thus, if you need to deserialize JSON containing a complex object (such as your serialized SqlGeography), you have the following options:

1. Create and deserialize to a typed DataTable.

2. Use DataTableConverter directly to populate a pre-existing DataTable with pre-allocated columns, as shown here.

3. Deserialize to a list of DTOs such as the following:

   public class TableRowDTO
   {
       [JsonConverter(typeof(SqlGeographyConverter))]
       public SqlGeography f1 { get; set; }
       public int id { get; set; }
   }
   

   Where SqlGeographyConverter is, as required, a custom JsonConverter for SqlGeography.

   And then do:

   var settings = new JsonSerializerSettings().AddSqlConverters();
   var list = JsonConvert.DeserializeObject<List<TableRowDTO>>(jsonString, settings);
   

Answer 2

Implementing Json converters is certainly one way to handle the inability to serialize SqlGeography in Json.Net, however I think a much simpler solution is to NOT serialize SqlGeography directly. Instead, create a custom class that contains all of the data from SqlGeography, and then serialize/deserialize that instead.

public class SqlGeographyResult
    {
        private byte[] bytes { get; }

        public SqlGeographyResult(SqlGeography geography)
        {
            bytes = geography.Serialize().Value;
        }

        public SqlGeography ToGeography()
        {
            return SqlGeography.Deserialize(new System.Data.SqlTypes.SqlBytes(bytes));
        }
    }