Django celery random raise SystemExit

Question

I have a celery task that handles copying large files (30 GB). The function looks like this:

def copy(self, src, dest):
        self.execute('cp %s %s &' % (src, dest))
        while self.get_file_size(src) != self.get_file_size(dest):
            time.sleep(3)

Now it seems that celery randomly (sometime it happens after 1 min sometime 15 min) raise a SystemExit execption with the following traceback:

Traceback (most recent call last):
 File "/some/path/tasks.py", line 51, in create_vm
   vm.create()
 File "/some/path/models.py", line 115, in create
   self.server.copy(self.template_path, self.vdi_path)
 File "/some/path/models.py", line 451, in copy
   time.sleep(3)
 File "/usr/local/lib/python2.7/dist-packages/billiard/common.py", line 95, in _shutdown_cleanup
   sys.exit(-(256 - signum))
 File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 279, in exit
   return _exit()
SystemExit

What can cause celery to raise such an exception?

Answer 1

The reason why celery raised a SystemExit exception is because celery restarted.

From the log:

Restarting celery worker (./manage.py celeryd)

And the reason why celery restarted is because it was initially started over ssh, and it got killed when ssh user disconnected.