Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Data migration of image model

Tags:

wagtail

Hi this is mostly a copy paste of a question asked in Google groups:

Thanks to Wagtail docs, I was able to understand how to build a custom image model, BUT, as I have a website with more than 500 contents, I don't want to mess the whole thing up with a bad data migration. In fact, I am not sure of which migration operation I should use here. I think that I should this one: https://docs.djangoproject.com/en/1.8/ref/migration-operations/#altermodeltable Can someone confirm this ?

Thanks a lot Regards

EDIT: provide some details about the migration

  • I created a custom image model in order to provide image URLs directly into Wagtail API, on the image endpoint (example: https://github.com/wagtail/wagtaildemo/blob/api-tweaks/demo/models.py#L784-L795)
  • As it is explained in wagtail docs, a data migration is required. But as I never been through this procedure, I just want to be sure to do it the right way

This is surely wagtail-specific, why I may have omitted some details. Sorry for that. And as I don't have much reputation, I can't provide more than 2 links a single post 😞

like image 262
fabienheureux Avatar asked Jan 30 '17 08:01

fabienheureux


People also ask

What is data model migration?

Data migration allows you to convert data from one model (schema) to another, using mappings. The migration process itself is discussed in The Migration Process. How you perform a migration is discussed in Initiating the Migration Process.

What is data migration with example?

Data migration is the process of transferring data from one storage system or computing environment to another. There are many reasons your enterprise might need to undertake a data migration project. For example, you might be replacing servers or storage devices or consolidating or decommissioning data center.

What is data migration diagram?

The purpose of the data migration diagram is to show the flow of data from the source to the target applications. The diagram will provide a visual representation of the spread of sources/targets and serve as a tool for data auditing and establishing traceability.


1 Answers

I recently did this same migration to my own customised model, however, we were not actively using Tags so I did not worry about transferring tags across. This migration does not delete the original Image database records as I wanted to keep them just in case.

Step 1 - Create Model app_name/models.py

from django.db import models
from wagtail.wagtailimages.models import (
    Image, AbstractImage, AbstractRendition)


class ExtendedImage(AbstractImage):
    caption = models.CharField(max_length=255, blank=True)

    admin_form_fields = Image.admin_form_fields + (
        'caption',
    )


class ExtendedRendition(AbstractRendition):
    image = models.ForeignKey(ExtendedImage, related_name='renditions')

    class Meta:
        unique_together = (
            ('image', 'filter_spec', 'focal_point_key'),
        )

Step 2 - Run Migrations Looks like you may have done this, it creates the customised model

  1. $python manage.py makemigrations
  2. $python manage.py migrate

Step 3 - Create Custom Data Migration

  1. $python manage.py makemigrations --empty app_name

  2. Edit that file as below (see comments inline)

```

from __future__ import unicode_literals

from django.db import migrations

# This only COPIES images from the existing model to the new one
# to reverse during testing - run
# ./manage.py migrate main 0036_auto_20170524_1811 (replace with file name of previous migration)


def forwards_func(apps, schema_editor):
    # We get the model from the versioned app registry;
    wagtail_image_model = apps.get_model('wagtailimages', 'Image')
    extended_image_model = apps.get_model('main', 'ExtendedImage')
    db_alias = schema_editor.connection.alias
    # Get images
    images = wagtail_image_model.objects.using(db_alias).all()
    new_images = []
    for image in images:
        new_images.append(extended_image_model(
            id=image.id,
            title=image.title,
            file=image.file,
            width=image.width,
            height=image.height,
            created_at=image.created_at,
            focal_point_x=image.focal_point_x,
            focal_point_y=image.focal_point_y,
            focal_point_width=image.focal_point_width,
            focal_point_height=image.focal_point_height,
            file_size=image.file_size,
            # image=test_image.caption,
            collection=image.collection,
            # tags=image.tags, # does not copy over
            uploaded_by_user=image.uploaded_by_user,
        ))
    # Create images in new model
    extended_image_model.objects.using(db_alias).bulk_create(new_images)
    # Leave all images in previous model


def reverse_func(apps, schema_editor):
    # We get the model from the versioned app registry;
    extended_image_model = apps.get_model('main', 'ExtendedImage')
    db_alias = schema_editor.connection.alias
    # Delete all images created in the new model
    extended_image_model.objects.using(db_alias).all().delete()


class Migration(migrations.Migration):

    dependencies = [
        ('main', '0036_auto_20170524_1811'), # Django will create this part
    ]

    operations = [
        migrations.RunPython(forwards_func, reverse_func),
    ]

```

Step 4 - Update Settings

WAGTAILIMAGES_IMAGE_MODEL = 'my_app.ExtendedImage'

Test this along the way and when you are ready you can delete the original image database rows if want.

** Note about Postgres One issue we ran into was Postgres did not like me migrating things to the primary key, we had to run a SQL query to reset the current key to the max + 1

like image 105
LB Ben Johnston Avatar answered Oct 09 '22 09:10

LB Ben Johnston