Laravel queue:restart is not killing workers

Question

I have a laravel 5.4 app deployed via envoyer to a non-forge server. I am running queue workers on the database driver, using supervisor to monitor, setup as described in the docs;

command=php /home/data/app/current/artisan queue:work --sleep=3 --tries=3

and using the envoyer deployment hook

cd ~/app/current
php artisan queue:restart

Problem is, after each deployment the queue workers are not restarted, the old ones continue to run and then throw errors because they are working on previous releases of the code. Running queue:restart manually from the CLI doesn't work either.

data@medicone:~/ccpbase/current$ ps -aux | grep queue:work
data      4347  0.0  0.2 292988 34852 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4348  0.0  0.2 292988 34864 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4349  0.0  0.2 292988 34720 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4350  0.0  0.2 292988 34880 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4351  0.0  0.2 292988 34972 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4382  0.0  0.2 292988 34904 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4383  0.0  0.2 292988 34992 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4384  0.0  0.2 292988 34980 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4863  0.0  0.0  14228  1016 pts/0    S+   11:32   0:00 grep queue:work
data@medicone:~/ccpbase/current$
data@medicone:~/ccpbase/current$ php artisan queue:restart
Broadcasting queue restart signal.
data@medicone:~/ccpbase/current$ ps -aux | grep queue:work
data      4347  0.0  0.2 292988 34852 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4348  0.0  0.2 292988 34864 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4349  0.0  0.2 292988 34720 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4350  0.0  0.2 292988 34880 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4351  0.0  0.2 292988 34972 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4382  0.0  0.2 292988 34904 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4383  0.0  0.2 292988 34992 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4384  0.0  0.2 292988 34980 ?        S    11:12   0:00 php /home/data/ccpbase/current/artisan queue:work --sleep=3 --tries=3
data      4869  0.0  0.0  14228   960 pts/0    S+   11:32   0:00 grep queue:work
data@medicone:~/ccpbase/current$

If I find and kill the 8 running processes manually, supervisor does restart them correctly and my queued jobs work again.

Can anybody think of anything that might be preventing these workers from being killed? There is nothing relevant in storage/logs/laravel.log

Answer 1

Its important to understant few points related to following queue:restart command

php artisan queue:restart

• This command will instruct all queue workers to gracefully "die" after they finish processing their current job so that no existing jobs are lost.
• The queue uses the cache to store restart signals, so you should verify a cache driver is properly configured for your application before using this feature.
• Since the queue workers will die when the queue:restart command is executed, you should be running a process manager such as Supervisor to automatically restart the queue workers.

Which means queue:restart will not immediately restart the queue but simply 'Broadcasting queue restart signal' for queue worker to get it die, once it finished the current assigned task.

Ref : https://laravel.com/docs/5.6/queues#queue-workers-and-deployment

Answer 2

In my case also, php artisan queue:restart did nothing. The queue workers' age confirmed that they were not restarting in response to that. The following will show you when your workers launched. Mine were all different, by several days.

$ ps -eo pid,lstart,cmd | grep queue:work

As part of my deployment script (gitlab, in my case) I am running php artisan queue:restart, so the queue workers restart after any code is pushed live.

As mentioned in a previous answer, the queue:restart command saves to the Laravel cache. I am using the 'file' cache driver, so cache entries are on disk at storage/framework/cache/data by default.

It turned out that my deploy script was running php artisan queue:restart as the gitlab-runner user, and thus any cache entries it created were also owned by that user.

Running the following instantly fixed my problem. The queue workers restarted the moment this was run:

sudo chown -R www-data:www-data storage/framework/cache

After this, commandline invocations of php artisan queue:restart work perfectly.

The long-term fix was to ensure my deploy script ran php artisan queue:restart as the proper user (or chown's the cache files as needed).

tldr; if using the 'file' cache driver, your web server user needs to be able to read/write the cache entry created by the php artisan queue:restart command. Check your permissions!