How to perform integration testing on EF Core migrations

Question

I've searched extensively on this subject and have found zero results for exactly what it is that I want to do.

From a high level, this is what I want to do:

• Arrange: Within a test method, call appropriate services to seed the database.
• Act: Run a migration to update the database schema.
• Assert: Data integrity, i.e., seed data can be accessed as per the updated entity structure.

What's the point of this?
Well it's pretty much described in the assertion above; data integrity. Migrations in this project are run on app startup using Database.Migrate(). I'd like to make sure that there's no loss/corruption of existing data.

All examples of integration testing I've come across pretty much run Database.Migrate() as part of the test setup, followed by seeding, and then making assertions. This however is only useful in testing the data access layer given the latest schema (with all migrations already applied). It's not useful in testing the effects of a specific migration on data that already exists.

Question:

How have others tackled the issue of testing data integrity across migrations? I'm looking for a setup which would work well with a CI pipeline.

Answer 1

Please see my other answer as it's newer and much simpler to the following approach.

One option that I can think of, would work as follows...
(Please note that I haven't tested this, it's just theory at this stage, so feel free to comment on where I may have gone wrong. I've made notes myself at the end addressing issues that I see; notes correspond to items tagged with a * or ^)

Assume you already have migrations IM, M1, M2, ..., Mn, where IM is the Initial Migration, and migration Mn is the last successfully tested migration. Now let's suppose you want to test a new migration Mn+1, that is a result of (among other things) updating the User entity to no longer have separate FirstName and LastName properties, but instead have a single Name property. Let's give it the name MergeFistAndLastNames.

Before proceeding with your test method, you'd have to capture the state of the System Under Test (SUT) exactly as it was, previous to applying migration Mn+1. Hence do the following:

1. Create a new test project for migrations, if one does not already exist.
2. Remove from the migrations test project all existing content^ (the content is described in the following steps).
3. Place into the migrations test project, a copy of all entities in the model corresponding to Mn, that need to be modified in order to generate Mn+1. Edit their names to be suffixed (for e.g) with Before_MergeFistAndLastNames. So since entity User needs to be updated, place a copy of it in the test project and name it User_Before_MergeFirstAndLastNames.
4. Do the same for any contexts* that need to be updated in order to generate Mn+1. So assuming (let's say) the MembershipContext needs to be updated, make a copy of it in the test project and call it MembershipContext_Before_MergeFirstAndLastNames.
5. Finally, do the same for any services you will need to call in order to seed the test database (if you make use of the Repository pattern, it would be your repository implementations that would typically be called to modify the database and a change to your entities may trigger a change to your repository implementations).

Now for your test method^^, you will:

• Arrange: Apply all migrations up to and including Mn using IMigrator.Migrate. Then call the required services to seed the test database**.
• Act: Using IMigrator.Migrate again, apply migration Mn+1.
• Assert: Data integrity***.

Notes:
*: If a context is copied to the test project, any DbSet properties within it that correspond to updated entities would need to be manually edited to use the copied entities. For instance, in the example above, if MembershipContext_Before_MergeFirstAndLastNames had a property DbSet<User> User {get; set;}, it would have to be changed to DbSet<User_Before_MergeFirstAndLastNames> User {get; set;}.

**: You may need to call more than one service. If you're using IoC everywhere, it should be relatively easy to inject the copied services.

***: A data integrity assertion really depends on the specific migration you are testing and its potential affects. For the example given above, it may be worthwhile to check that a user seeded using (the old way of) a separate first and last name can still be retrieved via the updated UserService which should return a User with Name equal to the concatenation of the first and last names.

^: The migrations test project needs to be cleared of all content when testing a new migration.
Informal proof: Let's assume this is not needed. Using the example above, this project would contain a test method with a reference to the User entity corresponding to migration Mn+1. Now let's say an update is made to User, which results in a newer migration Mn+2. The existing test method could potentially no longer compile, if it references User in a way that's no longer applicable. ∎

The above technique would work well in a CI setup where all previous migrations would already have been tested before being merged in to production. In such a setup, it would make sense to only ever be interested in testing the newest migration.

^^: For each test method that tests the effects of an Up migration, an inverse test method is required that tests the effects of the corresponding Down migration.