Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

supervisord always returns exit status 127 at WebFaction

I keep getting the following errors from supervisord at webFaction when tailing the log:

INFO exited: my_app (exit status 127; not expected)
INFO gave up: my_app entered FATAL state, too many start retries too quickly

Here's my supervisord.conf:

[unix_http_server]
file=/home/btaylordesign/tmp/supervisord.sock

[rpcinterface:supervisor]
supervisor.rpcinterface_factory=supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///home/btaylordesign/tmp/supervisord.sock

[supervisord]
logfile=/home/btaylordesign/tmp/supervisord.log
logfile_maxbytes=50MB
logfile_backups=5
loglevel=info
nodaemon=false
pidfile=/home/btaylordesign/tmp/supervisord.pid supervisord.pid

[program:my_app]
directory=/home/btaylordesign/webapps/my_app/my_app
command=celery worker -A my_app --concurrency=3 --loglevel=debug

I'm starting supervisord from the same directory as supervisord.conf:

$ supervisord -c ./supervisord.conf

but I can't seem to find the right combination of settings. I need to be able to do three things:

  1. Start my celery workers in the background and keep them running.
  2. Stop the celery workers when I deploy code.
  3. Restart the celery workers when the deploy is complete.

But, I can't do any of that until I resolve the error. What am I doing wrong?

like image 686
Brandon Avatar asked Feb 20 '15 02:02

Brandon


People also ask

Why is the exit code in supervisor 3x 0?

In Supervisor 3.x, very few conditions caused the exit code to be non-zero. For example, you could supervisorctl start zope and if zope failed to start, the exit code would still be 0. There were several tickets and many commenters requesting that supervisorctl be changed to set meaningful exit codes when possible.

Why is the shell throwing an exit code 127-unknown command?

If not, the shell will throw an exit code 127 - "Unknown command". 127 is a command not found. Once your server machine is up & you are not able to access server, you troubleshoot by checking logs from /var/log/syslog by the following command The issue is the given User cannot access the ExecStart within the PATH you have set.

What does 127 exit status mean in Docker?

The 127 exit status mean command not found but it's hard to say why you are getting that error without more info. I use Docker-Compose to get the code inside the container, it doesn't have much more than that and the file I am trying to execute actually exist because if I execute it directly the server goes up as expected.

Should Supervisord add is-enabled/is-available and is-failed status commands?

I think if supervisord wanted to model it's status command on systemctl then adding is-enabled, is-available, and is-failed would be good additions for the future. In the interim, if you wanted a safer behavior that detected failure cases at the status level, and don't mind stopped processes, ignoring 3 is maybe an option.


1 Answers

Exit code 127 means "command not found":

http://www.tldp.org/LDP/abs/html/exitcodes.html

Try passing the full path to the celery command:

command=/home/something/bin/celery worker -A my_app --concurrency=3 --loglevel=debug

Also, try setting the redirect_stderr and stdout_logfile options in your [program:x] section to capture the error message and make debugging easier.

like image 137
Steven Kryskalla Avatar answered Nov 15 '22 04:11

Steven Kryskalla