django: 'python manage.py migrate' taking hours (and other weird behavior)

Question

I made some changes to one of my tables in models.py and tried migrating it with 'python manage.py migrate,' and it's taking hours. I only changed the names of three field (column) names, and it's been running for over 2 hours now. It ran smoothly (I think) in a matter of minutes when I created the table earlier this morning. Start of season is the model where changes were made.

Here is what from models.py looks like now:

from django.db import models
from django.contrib.gis.db import models as gismodels
# from django.contrib.gis import admin

# Create your models here.

class Location(models.Model): # table name automatically chirps_location
    locationID = models.IntegerField(default=0, primary_key=True)
    lat = models.FloatField(default=0.0)
    lon = models.FloatField(default=0.0)
    geom = gismodels.PointField(null=True)
    objects = gismodels.GeoManager()
    def __unicode__(self):
        return u"LocationID: " + unicode(self.locationID)

# admin.site.unregister(Location)
# admin.site.register(Location, admin.OSMGeoAdmin)


class Rainfall(models.Model):
    location = models.ForeignKey(Location)
    year = models.IntegerField(default=0)
    pentad_num = models.IntegerField(default=0)        
    pentad_val = models.FloatField(default=0.0)

    class Meta:
        ordering = ['location', 'year', 'pentad_num']

    def __unicode__(self):
        return u"LocationID: " + unicode(self.location.locationID) + u" Year: " + unicode(self.year) + u" Pentad: " + unicode(self.pentad_num)


class Start_of_Season(models.Model):
    location = models.ForeignKey(Location)
    crop = models.CharField(default='', max_length=40)
    year = models.IntegerField(default=0)
    first_rain = models.IntegerField(default=0)
    onset_rain = models.IntegerField(default=0)
    start_season = models.IntegerField(default=0)

    class Meta:
        ordering = ['location', 'crop', 'year']    

    def __unicode__(self):
        return u"LocationID: " + unicode(self.location.locationID) + u" Year: " + unicode(self.year) + u" Start of Season pentad: " + unicode(self.start_season)

There was also some weird behavior that I couldn't explain going on before this, so I'll give a brief rundown of all of that too in case it's all related.

At first, my Start_of_Season class looked like this (notice the only difference is the names of the last 3 fields):

class Start_of_Season(models.Model):
    location = models.ForeignKey(Location)
    crop = models.CharField(default='', max_length=40)
    year = models.IntegerField(default=0)
    first_rain_pentad = models.IntegerField(default=0)
    onset_rain_pentad = models.IntegerField(default=0)
    start_season_pentad = models.IntegerField(default=0)

    class Meta:
        ordering = ['location', 'crop', 'year']    

    def __unicode__(self):
        return u"LocationID: " + unicode(self.location.locationID) + u" Year: " + unicode(self.year) + u" Start of Season pentad: " + unicode(self.start_season)

Migrating it with:

python manage.py makemigrations
python manage.py migrate

appeared to run smoothly.

But when I ran a python script (excerpt) to add rows to this newly created Start_of_Season table:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "access.settings")
from chirps.models import Location, Rainfall, Start_of_Season
django.setup()

try:
    with transaction.atomic():
        for loc in Location.objects.all():
            locID = loc.locationID
            for yr in range(1981, 2014):
                # some computations to find: c1, c2, c3
                start_of_season = Start_of_Season()
                start_of_season.location = Location.objects.get(locationID = locID)
                start_of_season.crop = 'millet'
                start_of_season.year = yr
                start_of_season.first_rain_pentad = c1
                start_of_season.onset_rain_pentad = c2
                start_of_season.start_season_pentad = c3
                start_of_season.save()

At first, the rows were never added to the database (according to psql at least). Then I got the error "start_of_season has no attribute start_season" which is weird because I never tried to access that attribute in my script, only "start_of_season.start_season_pentad"

So then I thought, okay, I'll change the names of the fields so that start_of_season does have that attribute. And that's when I edited models.py to look like the excerpt at the top of the post.

After updating updating models.py,

python manage.py makemigrations

ran with no errors:

(access_mw)mwooten@ip-10-159-67-226:~/dev/access$ python manage.py makemigrations
Did you rename start_of_season.first_rain_pentad to start_of_season.first_rain (a IntegerField)? [y/N] y
Did you rename start_of_season.onset_rain_pentad to start_of_season.onset_rain (a IntegerField)? [y/N] y
Did you rename start_of_season.start_season_pentad to start_of_season.start_season (a IntegerField)? [y/N] y
Migrations for 'chirps':
  0010_auto_20150901_1454.py:
    - Rename field first_rain_pentad on start_of_season to first_rain
    - Rename field onset_rain_pentad on start_of_season to onset_rain
    - Rename field start_season_pentad on start_of_season to start_season

, but:

python manage.py migrate

has been stuck here for hours:

(access_mw)mwooten@ip-10-159-67-226:~/dev/access$ python manage.py migrate
Operations to perform:
  Synchronize unmigrated apps: staticfiles, gis, djgeojson, messages, leaflet
  Apply all migrations: admin, chirps, contenttypes, auth, sessions
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying chirps.0010_auto_20150901_1454...

I don't see why this should take so long, seeing as how creating the actual table didn't. I'm also not sure why I was getting the other errors. Any ideas as to what's going on?

Thanks

Answer 1

More often than not this is because there is another SQL client or django server or shell reading/writing to the table that you are trying to modify.

An SQL command that changes a schema cannot be successfully executed while a table is being accessed. Ideally an error message should pop up but what usually happens is that the command just waits for the others to finish.

Once you close all your open shells, and sql clients, the schema change can happen. In the worst case, you might need to temporarily take the site offline too.