Change type of Django model field from CharField to ForeignKey

Question

I need to change the type of a field in one of my Django models from CharField to ForeignKey. The fields are already populated with data, so I was wondering what is the best or right way to do this. Can I just update the field type and migrate, or are there any possible 'gotchas' to be aware of? N.B: I just use vanilla Django management operations (makemigrations and migrate), not South.

Answer 1

This is likely a case where you want to do a multi-stage migration. My recommendation for this would look something like the following.

First off, let's assume this is your initial model, inside an application called discography:

from django.db import models

class Album(models.Model):
    name = models.CharField(max_length=255)
    artist = models.CharField(max_length=255)

Now, you realize that you want to use a ForeignKey for the artist instead. Well, as mentioned, this is not just a simple process for this. It has to be done in several steps.

Step 1, add a new field for the ForeignKey, making sure to mark it as null:

from django.db import models

class Album(models.Model):
    name = models.CharField(max_length=255)
    artist = models.CharField(max_length=255)
    artist_link = models.ForeignKey('Artist', null=True)

class Artist(models.Model):
    name = models.CharField(max_length=255)

...and create a migration for this change.

./manage.py makemigrations discography

Step 2, populate your new field. In order to do this, you have to create an empty migration.

./manage.py makemigrations --empty --name transfer_artists discography

Once you have this empty migration, you want to add a single RunPython operation to it in order to link your records. In this case, it could look something like this:

def link_artists(apps, schema_editor):
    Album = apps.get_model('discography', 'Album')
    Artist = apps.get_model('discography', 'Artist')
    for album in Album.objects.all():
        artist, created = Artist.objects.get_or_create(name=album.artist)
        album.artist_link = artist
        album.save()

Now that your data is transferred to the new field, you could actually be done and leave everything as is, using the new field for everything. Or, if you want to do a bit of cleanup, you want to create two more migrations.

For your first migration, you will want to delete your original field, artist. For your second migration, rename the new field artist_link to artist.

This is done in multiple steps to ensure that Django recognizes the operations properly. You could create a migration manually to handle this, but I will leave that to you to figure out.

Answer 2

Adding on top of Joey's answer, detailed steps for Django 2.2.11.

Here are the models from my use case, that consists of a Company and Employee model. We have to convert designation to a foreign key field. The app name is called core

class Company(CommonFields):
    name = models.CharField(max_length=255, blank=True, null=True

class Employee(CommonFields):
    company = models.ForeignKey("Company", on_delete=models.CASCADE, blank=True, null=True)
    designation = models.CharField(max_length=100, blank=True, null=True)

Step 1

Create a foreign key designation_link in Employee and mark it as null=True

class Designation(CommonFields):
    name = models.CharField(max_length=255)
    company = models.ForeignKey("Company", on_delete=models.CASCADE, blank=True, null=True)

class Employee(CommonFields):
    company = models.ForeignKey("Company", on_delete=models.CASCADE, blank=True, null=True)
    designation = models.CharField(max_length=100, blank=True, null=True)
    designation_link = models.ForeignKey("Designation", on_delete=models.CASCADE, blank=True, null=True)

Step 2

Create empty migration. Using the command:

python app_code/manage.py makemigrations --empty --name transfer_designations core

This will create a following file in migrations directory.

# Generated by Django 2.2.11 on 2020-04-02 05:56

from django.db import migrations


class Migration(migrations.Migration):

    dependencies = [
        ('core', '0006_auto_20200402_1119'),
    ]

    operations = [
    ]

Step 3

Populate the empty migration with a function that loops over all Employees, creates a Designation and links it to the Employee.

In my use case each Designation is also linked to a Company. Which means that Designation may contain two rows for "managers", one for company A, another for company B.

Final migration would look something like this:

# core/migrations/0007_transfer_designations.py

# Generated by Django 2.2.11 on 2020-04-02 05:56

from django.db import migrations

def link_designation(apps, schema_editor):
    Employee = apps.get_model('core', 'Employee')
    Designation = apps.get_model('core', 'Designation')
    for emp in Employee.objects.all():
        if(emp.designation is not None and emp.company is not None):
            desig, created = Designation.objects.get_or_create(name=emp.designation, company=emp.company)
            emp.designation_link = desig
            emp.save()

class Migration(migrations.Migration):

    dependencies = [
        ('core', '0006_auto_20200402_1119'),
    ]

    operations = [
        migrations.RunPython(link_designation),
    ]

Step 4

Finally run this migration using:

python app_code/manage.py migrate core 0007

Answer 3

That's a continuation of the great answer by Joey. How to rename the new field to the original name?

If the field has data, it probably means that you are using it elsewhere in your project, therefore this solution will leave you with a field named differently, and you have to either refactor the project to use the new field or delete the old field and rename the new one.

Be aware that this process is not going to prevent you to refactor code. If you where using a CharField with CHOICES, you were accessing its content with get_filename_display(), for example.

If you try to delete the field to make a migration, for then renaming the other field and make another migration, you'll see Django complaining because you cannot delete a field that you are using in the project.

Just create an empty migration as Joey explained, and put this in operations:

operations = [
    migrations.RemoveField(
        model_name='app_name',
        name='old_field_name',
    ),
    migrations.RenameField(
        model_name='app_name',
        old_name='old_field_name_link',
        new_name='old_field_name',
    ),
]

Then run migrate and you'll have the changes made in your database, but obviously not in your model, it's time now to delete the old field and to rename new ForeignKey field to the original name.

I don't think that doing this is particularly hacky, but still, only do this kind of things if you are fully understanding what are you messing with.