Data Encryption in Data Layer with ASP.NET Core Entity Framework

Question

I am currently designing a web application where the data needs to be stored encrypted.

Planned technologies used:

• ASP.NET Core API
• ASP.NET Core Entity Framework
• MS SQL Server 2012
• any Web Frontend
• Because of the specification, we need to store all data encrypted in the database.

Which would be a good approach to achieve this while still be able to use the Entity Framework & LINQ, so the developer does not have to take care of the encryption.

Is it possible to encrypt the whole database?

Answer 1

First, thanks to @Steven as my answer is based on that.

This solution extends what his solution has by adding some further configuration of the IDataProtectionProvider so as to enable migrations using the Package Manager Console in Visual Studio. Also, I have selected to use varbinary as the backing data type in SQL. Further, I have selected to store the encryption keys in the server runtime directory (so make sure you back them up if you do this).

Program.cs

Note that this solution was built using .NET 6.0, and so there is no longer a Startup.cs and Program.cs, rather only Program.cs. If you are using an older template this should still work, you will just need to move some of the startup things around.

using Microsoft.AspNetCore.DataProtection;
using Microsoft.Extensions.DependencyInjection;
using System;
using System.IO;

var keyDirectory = Path.Combine(AppContext.BaseDirectory, "Keys");
Directory.CreateDirectory(keyDirectory);

builder.Services.AddDataProtection()
    .SetApplicationName("My App Name")
    .PersistKeysToFileSystem(new DirectoryInfo(keyDirectory));

builder.Services.AddDbContext<MyDbContext>(options => options.UseSqlServer("My SQL connection string"));

MyModel.cs

Just any model class. You can decorate it with attributes and everything you would normally do for Entity Framework.

public class MyModel 
{
    public string EncryptedProperty { get; set; }
}

EncryptedConverter.cs

This is where the magic happens, it will run between your application code and the database to make the encryption process entirely transparent.

using Microsoft.AspNetCore.DataProtection;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using System.Text.Json;

/// <summary>
/// Converts string values to and from their underlying encrypted representation
/// </summary>
public class EncryptedConverter : EncryptedConverter<string>
{
    public EncryptedConverter(IDataProtectionProvider dataProtectionProvider) : base(dataProtectionProvider) { }
}

/// <summary>
/// Converts property values to and from their underlying encrypted representation
/// </summary>
/// <typeparam name="TProperty">The property to encrypt or decrypt</typeparam>
public class EncryptedConverter<TProperty> : ValueConverter<TProperty, byte[]>
{
    private static readonly JsonSerializerOptions? options;

    public EncryptedConverter(IDataProtectionProvider dataProtectionProvider) :
        base(
            x => dataProtectionProvider.CreateProtector("encryptedProperty").Protect(JsonSerializer.SerializeToUtf8Bytes(x, options)),
            x => JsonSerializer.Deserialize<TProperty>(dataProtectionProvider.CreateProtector("encryptedProperty").Unprotect(x), options),
            default
        )
    { }
}

MyDbContext.cs

The main difference here is adding another constructor to enable usage of encrypted properties during migrations. This also allows seeding database values using the builder.Entity<MyModel>().HasData() method. The seeded values will encrypt correctly.

using Microsoft.AspNetCore.DataProtection;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using System;
using System.IO;

public class MyDbContext : DbContext 
{
    private IDataProtectionProvider dataProtectionProvider;

    /// <summary>
    /// For migrations
    /// </summary>
    public MyDbContext() 
    {
        // Note that this should match your options in Program.cs
        var info = new DirectoryInfo(Path.Combine(AppContext.BaseDirectory, "Keys"));

        var provider = DataProtectionProvider.Create(info, x =>
        {
            x.SetApplicationName("My App Name");
            x.PersistKeysToFileSystem(info);
        });

        dataProtectionProvider = provider;
    }

    public MyDbContext(DbContextOptions<MyDbContext> options, IDataProtectionProvider dataProtectionProvider) : base(options)
    {
        this.dataProtectionProvider = dataProtectionProvider;
    }

    public DbSet<MyModel> MyModel { get; set; }

    protected override void OnModelCreating(ModelBuilder builder) 
    {
        builder.Entity<MyModel>(model => 
        {
            model.Property(x => x.EncryptedProperty)
                .HasColumnType("varbinary(max)")
                .HasConversion(new EncryptedConverter(dataProtectionProvider));
        });
    }
}

Answer 2

A good approach would be to encrypt your data when saving changes to your database, and decrypt when reading you data from the database.

I developed a library to provide encrypted fields within an Entity Framework Core context.

You can use my EntityFrameworkCore.DataEncryption plugin to encrypt your string fields when saving changes using a built-in or custom encryption provider. Actually, only the AesProvider has been developed.

To use it, simply add the [Encrypted] attribute to your string properties of your Model and then override the OnModelCreating() method in your DbContext class, and then call the modelBuilder.UseEncryption(...) by passing it an encryption provider (AesProvider or any class that inherits from IEncryptionProvider.)

public class UserEntity
{
    public int Id { get; set; }

    [Encrypted]
    public string Username { get; set; }

    [Encrypted]
    public string Password { get; set; }

    public int Age { get; set; }
}

public class DatabaseContext : DbContext
{
    // Get key and IV from a Base64String or any other ways.
    // You can generate a key and IV using "AesProvider.GenerateKey()"
    private readonly byte[] _encryptionKey = ...; 
    private readonly byte[] _encryptionIV = ...;
    private readonly IEncryptionProvider _provider;

    public DbSet<UserEntity> Users { get; set; }

    public DatabaseContext(DbContextOptions options)
        : base(options)
    {
        this._provider = new AesProvider(this._encryptionKey, this._encryptionIV);
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.UseEncryption(this._provider);
    }
}

Results on saving:

Hope it helps.

Answer 3

First of all, don't confuse encrypting with hashing, in Eastrall's answer they imply that you could use encryption for a password field. Do not do this

Also, you should change the initialisation vector every time you encrypt a new value, which means you should avoid implementations like Eastrall's library that set a single IV for the whole database.

Modern encryption algorithms are designed to be slow, so encrypting everything in your database is going to affect your performance at least marginally.

If done properly, your encrypted payload is not going to just be the cipher text, but should also contain the ID of the encryption key, details about the algorithm used, and a signature. This means your data is going to take up a lot more space compared to the plain text equivalent. Take a look at https://github.com/blowdart/AspNetCoreIdentityEncryption if you want to see how you could implement that yourself. (The readme in that project is worth reading anyway)

With that in mind, the best solution for your project might depend on how important it is for you to minimise those costs.

If you're going to use the .NET Core Aes.Create(); like in the library in Eastrall's answer, the cipher text is going to be a byte[] type. You could use the column type in your database provider for byte[], or you could encode as base64 and store as a string. Typically storing as a string is worthwhile: base64 will take up about 33% more space than byte[], but is easier to work with.

I suggest making use of the ASP.NET Core Data Protection stack instead of using the Aes classes directly, as it helps you do key rotation and handles the encoding in base64 for you. You can install it into your DI container with services.AddDataProtection() and then have your services depend upon IDataProtectionProvider, which can be used like this:

// Make sure you read the docs for ASP.NET Core Data Protection!

// protect
var payload = dataProtectionProvider
    .CreateProtector("<your purpose string here>")
    .Protect(plainText);

// unprotect
var plainText = dataProtectionProvider
    .CreateProtector("<your purpose string here>")
    .Unprotect(payload);

Of course, read the documentation and don't just copy the code above.

---

In ASP.NET Core Identity, the IdentityUserContext uses a value converter to encrypt personal data marked with the [ProtectedPersonalData] attribute. Eastrall's library is also using a ValueConverter.

This approach is handy because it doesn't require you to write code in your entities to handle conversion, something that might not be an option if you are following a Domain Driven Design approach (e.g. like the .NET Architecture Seedwork).

But there is a drawback. If you have a lot of protected fields on your entity. The code below would cause every single encrypted field on the user object to get decrypted, even though not a single one is being read.

var user = await context.Users.FirstOrDefaultAsync(u => u.Id == id);
user.EmailVerified = true;
await context.SaveChangesAsync();

You could avoid using a value converter by instead using a getter and setter on your property like the code below. However that means you will need to place encryption specific code in your entity, and you will have to wire up access to whatever your encryption provider is. This could be a static class, or you'll have to pass it in somehow.

private string secret;

public string Secret {
  get => SomeAccessibleEncryptionObject.Decrypt(secret);
  set => secret = SomeAccessibleEncryptionObject.Encrypt(value);
}

You would then be decrypting every time you access the property, which could cause you unexpected trouble elsewhere. For example the code below could be very costly if emailsToCompare was very large.

foreach (var email in emailsToCompare) {
  if(email == user.Email) {
    // do something...
  }
}

You can see that you'd need to memoize your encrypt and decrypt calls, either in the entity itself or in the provider.

Avoiding the value converter while still hiding the encryption from outside the entity or the database configuration is complex. And so if performance is so much of an issue that you can't go with the value converters, then your encryption is possibly not something that you can hide away from the rest of your application, and you would want to be running the Protect() and Unprotect() calls in code completely outside of your Entity Framework code.

---

Here is an example implementation inspired by the value converter setup in ASP.NET Core Identity but using an IDataProtectionProvider instead of IPersonalDataProtector:

public class ApplicationUser
{
    // other fields...

    [Protected]
    public string Email { get; set; }
}

public class ProtectedAttribute : Attribute
{
}

public class ApplicationDbContext : DbContext
{
    public ApplicationDbContext(DbContextOptions options)
        : base(options)
    {
    }

    public DbSet<ApplicationUser> Users { get; set; }

    protected override void OnModelCreating(ModelBuilder builder)
    {
        // other setup here..
        builder.Entity<ApplicationUser>(b =>
        {
            this.AddProtecedDataConverters(b);
        });
    }

    private void AddProtecedDataConverters<TEntity>(EntityTypeBuilder<TEntity> b)
        where TEntity : class
    {
        var protectedProps = typeof(TEntity).GetProperties()
            .Where(prop => Attribute.IsDefined(prop, typeof(ProtectedAttribute)));

        foreach (var p in protectedProps)
        {
            if (p.PropertyType != typeof(string))
            {
                // You could throw a NotSupportedException here if you only care about strings
                var converterType = typeof(ProtectedDataConverter<>)
                    .MakeGenericType(p.PropertyType);
                var converter = (ValueConverter)Activator
                    .CreateInstance(converterType, this.GetService<IDataProtectionProvider>());

                b.Property(p.PropertyType, p.Name).HasConversion(converter);
            }
            else
            {
                ProtectedDataConverter converter = new ProtectedDataConverter(
                    this.GetService<IDataProtectionProvider>());

                b.Property(typeof(string), p.Name).HasConversion(converter);
            }
        }
    }

    private class ProtectedDataConverter : ValueConverter<string, string>
    {
        public ProtectedDataConverter(IDataProtectionProvider protectionProvider)
            : base(
                    s => protectionProvider
                        .CreateProtector("personal_data")
                        .Protect(s),
                    s => protectionProvider
                        .CreateProtector("personal_data")
                        .Unprotect(s),
                    default)
        {
        }
    }

    // You could get rid of this one if you only care about encrypting strings
    private class ProtectedDataConverter<T> : ValueConverter<T, string>
    {
        public ProtectedDataConverter(IDataProtectionProvider protectionProvider)
            : base(
                    s => protectionProvider
                        .CreateProtector("personal_data")
                        .Protect(JsonSerializer.Serialize(s, default)),
                    s => JsonSerializer.Deserialize<T>(
                        protectionProvider.CreateProtector("personal_data")
                        .Unprotect(s),
                        default),
                    default)
        {
        }
    }
}

Finally, the responsibility of encryption is complex and I would recommend ensuring you have a firm grasp of whatever setup you go with to prevent things like data loss from losing your encryption keys. Also, the DotNet Security CheatSheet from the OWASP Cheatsheet Series is a useful resource to read.