Retroactively set new ManyToManyField default values to existing model

Question

I have a Django model (called BiomSearchJob) which is currently live and I want to add a new many-to-many relation to make the system more customizable for the user. Previously, users can submit a job without specifying a set of TaxonomyLevelChoices but to add more features to the system, users should now be able to select their own taxonomy levels.

Here's the model:

class TaxonomyLevelChoice(models.Model):
    taxon_level = models.CharField(
        verbose_name="Taxonomy Chart Level", max_length=60)
    taxon_level_proper_name = models.CharField(max_length=60)

    def __unicode__(self):
        return self.taxon_level_proper_name

class BiomSearchJob(models.Model):
    ...
    # The new many-to-many relation
    taxonomy_levels = models.ManyToManyField(
        'TaxonomyLevelChoice', blank=False, max_length=3,
        default=["phylum", "class", "genus"])

    name = models.CharField(
        null=False, blank=False, max_length=100, default="Unnamed Job",
        validators=[alphanumeric_spaces])
    ...

Currently, all existing BiomSearchJobs implicitly have the three taxonomy levels listed in the default= term (which are not user-selectable) and hence are all the same in the database. After running migrate, I find that the previous jobs don't immediately have the three taxonomy level relations, they only return an empty set upon calling job.taxonomy_levels.all() (if job were an instance of BiomSearchJob).

Is there a way to retroactively add this relationship without manually going through everything? Ideally, by just running migrate I would like the existing BiomSearchJobs to have phylum, class, and genus listed in the taxonomy_levels attribute.

Answer 1

I think you're looking for a data migration, which is a migration allowing for data-only changes over the database.

You can create it this way:

python manage.py makemigrations <your app> --name=retroactively_add_levels

Then insert this code into the migration file just created:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import migrations


def add_taxonomy_levels(apps, schema_editor):
    BiomSearchJob = apps.get_model('<your app>', 'BiomSearchJob')
    TaxonomyLevelChoice = apps.get_model('<your app>', 'TaxonomyLevelChoice')
    for job in BiomSearchJob.objects.all():
        for choice in TaxonomyLevelChoice.objects.filter(taxon_level_proper_name__in=["phylum", "class", "genus"]):
            job.taxonomy_levels.add(choice)

class Migration(migrations.Migration):

   dependencies = []

operations = [
    migrations.RunPython(add_taxonomy_levels, reverse_code=migrations.RunPython.noop)
]

It works pretty much as an SQL query would do but it leverages on Django ORM.

Hope this helps.

Answer 2

Your approach cannot work, as you implicitly want to make a query on an instance property: Django can't guess that.

From Django's Doc, default can be a function, while

For fields like ForeignKey that map to model instances, defaults should be the value of the field they reference (pk unless to_field is set) instead of model instances.

So.... Either you pass a PK (eg ID) array, either you use a function to get queryset.

class TaxonomyLevelChoice(models.Model):
    taxon_level = models.CharField(
        verbose_name="Taxonomy Chart Level", max_length=60)
    taxon_level_proper_name = models.CharField(max_length=60)

    def __unicode__(self):
        return self.taxon_level_proper_name

def get_default_taxonomy_levels():
    ...
    return YourQuerySet


class BiomSearchJob(models.Model):
    ...
    # The new many-to-many relation
    taxonomy_levels = models.ManyToManyField(
        'TaxonomyLevelChoice', blank=False, max_length=3,
        default=get_taxonomy_levels_default)

    name = models.CharField(
        null=False, blank=False, max_length=100, default="Unnamed Job",
        validators=[alphanumeric_spaces])
    ...

I guess you will have migration issues if you migrate before TaxonomyLevelChoice instances has not been created.

I'd go for the function solution due to previous sentence, with some pseudo cache method though, as making a query each time you create BiomSearchJob is not an acceptable solution.

I'd do:

DEFAULT_TAXONOMY_LEVELS = None

def get_default_taxonomy_levels():
    if DEFAULT_TAXONOMY_LEVELS:
        return DEFAULT_TAXONOMY_LEVELS
    ...
    DEFAULT_TAXONOMY_LEVELS = YourQuerySet
    return YourQuerySet

Edit: As the question is to retro-actively set a Many to Many default value, I'd suggest a command to do so on existing instances, as I don't think migration will handle that for you.