Symfony messenger workers do not stop when running under supervisor

Question

I am facing an odd behavior of the Symfony Messenger component. I set it up according to the documentation and I am issuing a messenger:stop-workers signal on each deploy as instructed here. However, a bug occurred in our system which I traced back to the fact that an old version of the code was being used by the Messenger workers.

Upon some more investigation, this is what happens in our setup:

• A worker is running, managed by supervisor.
• Just for debugging of this particular case, I start a new worker in the terminal app/console messenger:consume --env=prod -vv async to see what happens
• I issue the stop command app/console messenger:stop-workers --env=prod
• I would now expect that both the workers would be stopped (and supervisor would restart the one it's handling). However, that does not happen. The "debugging" worker does stop, but the one running under supervisor does nothing.

The supervisor-managed workers are limited to 1 hour of process time, after which they are stopped and restarted. I can see in the supervisord.log that this works well. Every hour there are log entries about the processes stopping and starting. But there is nothing whatsoever about them stopping from the messenger:stop-workers command.

I'm looking for ideas on why this would happen. I read the implementation of the workers and the shutdown signal is sent through a cache, but I did not find any problems in our configuration of it.

Answer 1

I was running into a similar problem.

To force the stop of the rest of the consumers, as you have seen, the command uses a cache pool. In my case (and probably yours too), is the filesystem pool which is stored into /your_symfony_app/var/cache/{env}/pools

So if you are using Deployer or any other deployment system that replaces a symbolic link with every new deployment, you need to execute the command messenger:stop-workers inside the folder of your previous release.

Another option is to configure a cache pool shared by all the releases, like memcached or redis.

In my case, using Deployer (with a software still in development) I have been able to solve it by declaring a task like this, and putting it inside the deploy main task:

task('messenger:stop', function () {
    if (has('previous_release')) {
        run('{{bin/php}} {{previous_release}}/bin/console messenger:stop-workers');
    }
})->desc('Stop workers');