Custom JSON serialization of sensitive data for PCI compliance

Question

We have to log incoming requests and outgoing responses for our web service. This includes JSON serialization of each object so they can be stored in a database.

Some information is considered sensitive (such as Social Security Numbers, credit card numbers, etc.) and we cannot include these in our logs per PCI compliance. Right now we're manually replacing the values with a placeholder value (e.g. "[PRIVATE]") but this only works with string properties. Some data, such as a Date of Birth is not stored as a string so this doesn't work as the replacement of the property value happens before the serialization. The big problem is that it is too easy for someone to forget to do remove the sensitive data before logging it, which is highly undesirable.

To remedy this, I was thinking of creating a custom attribute and placing it on the property and then having the JSON serialization routine look for this attribute on each property and if it exists, replace the serialized value with a placeholder such as "[PRIVATE]".

Right now we are using the System.Web.Script.Serialization.JavaScriptSerializer for our serialization. Obviously it knows nothing of my custom attribute. How would I go about changing the serialization process so any data decorated with my custom "SensitiveData" attribute is replaced with a placeholder value? I'm not against using a different serializer but was hoping I could leverage the features of an existing one instead of writing my own.

Answer 1

Here's my solution, although it may need minor tweaks:

My custom JsonConverter:

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
using System.Linq;
using System.Reflection;

public class SensitiveDataJsonConverter : JsonConverter
{
    private readonly Type[] _types;

    public SensitiveDataJsonConverter(params Type[] types)
    {
      _types = types;
    }

    public override bool CanConvert(Type objectType)
    {
        return _types.Any(e => e == objectType);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var jObject = new JObject();
        var type = value.GetType();

        foreach (var propertyInfo in type.GetProperties())
        {
            // We can only serialize properties with getters
            if (propertyInfo.CanRead)
            {
                var sensitiveDataAttribute = propertyInfo.GetCustomAttribute<SensitiveDataAttribute>();
                object propertyValue;

                if (sensitiveDataAttribute != null)
                    propertyValue = "[REDACTED]";
                else
                    propertyValue = propertyInfo.GetValue(value);

                if (propertyValue == null)
                  propertyValue = string.Empty;

                var jToken = JToken.FromObject(propertyValue, serializer);

                jObject.Add(propertyInfo.Name, jToken);
            }
        }

        jObject.WriteTo(writer);
    }

Here's my custom attribute:

using System;

[AttributeUsage(AttributeTargets.Parameter | AttributeTargets.Property)]
public class SensitiveDataAttribute : Attribute
{

}

We use it like this:

string serialized;

try
{
    serialized = JsonConvert.SerializeObject(value, new SensitiveDataJsonConverter(value.GetType()));
}
catch // Some objects cannot be serialized
{
    serialized = $"[Unable to serialize object '{key}']";
}

Here's a test class I try to serialize:

class Person
{
    public Person()
    {
        Children = new List<Person>();
    }

    public List<Person> Children { get; set; }

    [SensitiveData]
    public DateTime DateOfBirth { get; set; }

    public string FirstName { get; set; }

    public string LastName { get; set; }

    [SensitiveData]
    public string SocialSecurityNumber { get; set; }

    public Person Spouse { get; set; }
}

This seemed to work great until I added the Spouse and Children properties but I was getting a NullReferenceException. I added this to the WriteJson method which corrected the problem:

if (propertyValue == null)
    propertyValue = string.Empty;

Answer 2

I was also looking for a way to hide GDPR data in Audit information. So i came across this question.

I've had a lot of issues implementing this JsonConvertor. I had a lot of issues with types and certainly with child types (i was serializing the AuditEvent from Audit.NET lib, and then some child props of that). I even added the code from @Duu82, but then still a lot of issues.

I however found another way of resolving my problem, and it was in another question/answer on SO: Replace sensitive data value on JSON serialization

So for future users who come across this, I found that answer more practical, less code and working