Using Django bulk_create gives UNIQUE constraint failed:solgeo_hourly.id

Question

I have two models, namely Project and Hourly. Whenever a new project is created, I want to also generate 8760 new Hourly instances.

There are two methods, one is very costly since 8760 times a new row is written to the database, i.e.:

for i in range(0, 24):
    clock_time=i*3600
    hourly = Hourly.objects.create(project=project, clock_time=clock_time)
    hourly.save()

The other method is by using the Django bulk_create(hourlys) method, which works fine whenever I use the default Hourly() constructor. However I want to use my custom Hourly.objects.create() function. This however gives an UNIQUE constraint error for hourly.id, i.e.:

class HourlyManager(models.Manager):
    """"""
    def create(self, project, clock_time):
        hourly = super().create(project=project, clock_time=clock_time)
        hourly.local_civil_time = services.calc_local_civil_time(clock_time, 
                                                                 project.TMZ, 
                                                                 project.lon)
        hourly.save()
        return hourly


class ProjectManager(models.Manager):
    """"""
    def create(self, owner, project_name, TMZ, lat, lon):
        project = super().create(owner=owner, project_name=project_name, TMZ=TMZ, lat=lat, lon=lon)
        project.save()

        hourlys = []
        for i in range(0, 24):
            hourlys.append(Hourly.objects.create(project=project, clock_time=clock_time))
        Hourly.objects.bulk_create(hourlys)

        return project


class Project(models.Model):
    objects = ProjectManager()

    owner           = models.ForeignKey('auth.User', related_name='projects', on_delete=models.CASCADE)
    project_name    = models.CharField(max_length=200)
    TMZ             = models.FloatField(default=0)
    lat             = models.FloatField(default=0)  # Radians
    lon             = models.FloatField(default=0)  # Radians


class Hourly(models.Model):
    objects = HourlyManager()

    project             = models.ForeignKey(Project, related_name='hourlys', on_delete=models.CASCADE)
    clock_time          = models.FloatField(default=0) 
    local_civil_time    = models.FloatField(default=0)

Answer 1

@arne, this is the wrong way to bulk_create.

    hourlys = []
    for i in range(0, 24):
        hourlys.append(Hourly.objects.create(project=project, 
     clock_time=clock_time))
     Hourly.objects.bulk_create(hourlys)

The proper way is you pass the attributes to the Hourly object creating a list of that. A compressed version would look like this here below.

hourlys = [Hourly(project=project, clock_time=i*3600) for i in range(24)]
hourlys_objects = Hourly.objects.bulk_create(hourlys)

See link here for more

As explained above when you call Hourly.objects.create the object is created (with a hit to the database also defeating the purpose of using bulk_create optimization).