Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Running a python script in virtual environment with node.js pm2

I would like to reference this question because I am certain that someone will flag this as a duplicate.

I am not looking for another reference to supervisord. I'm sure that it is great and all, but the node PM2 has the functionality that I require and is more straightforward to implement and test.

Manual Start

During prototyping, I created a virtual environment called 'p3env'. At the top of each script, I place a bash directive:

#!./py3env/bin/python

This allows me to execute each script in the directory using this particular python environment without having to activate it. It is very convenient and useful and the python script works well when I start it by hand.

I should be clear about what I mean when I say 'start it by hand'. My script is called 'strain_to_db.py'. When I start it by hand, I am on the shell via ssh:

./strain_to_db.py

This gets everything working that I need to have working.

PM2 Start

Moving from relative to absolute paths

To get pm2 working, I started with:

pm2 start ./strain_to_db.py

Specifying the Interpreter

Apparently pm2 ignores the directive at the top of the python script and attempts to execute using the global 'python'. No problem, I can specify the interpreter:

pm2 start ./strain_to_db.py --interpreter /home/ubuntu/db_if/p3env/bin/python

No dice. Again, maybe try more absolute paths:

pm2 start /home/ubuntu/db_if/strain_to_db.py --interpreter /home/ubuntu/db_if/p3env/bin/python

Running script as command-line option

Now I'm getting frustrated. I try another tactic. I attempt to run the python executable in the command line using:

/home/ubuntu/db_if/p3env/bin/python /home/ubuntu/db_if/strain_to_db.py

This works fine when pm2 isn't involved. When I try to pass this to pm2 using the 'command line argument' style:

pm2 start /home/ubuntu/db_if/p3env/bin/python -- /home/ubuntu/db_if/strain_to_db.py

Frustration

Same error. The error is always 'can't import pymysql', which is only installed on the virtual environment.

I am not sure where else to go with this. I have several scripts that I want to add to the pm2 execution monitor, but I can't seem to get one of them to start and run correctly.

like image 818
slightlynybbled Avatar asked Mar 18 '16 17:03

slightlynybbled


People also ask

Can PM2 run Python script?

Running Your Python Script with PM2By default, PM2 will run your script with Python when you start any . py file. However, if you have several versions of Python installed in your machine, you may select a specific Python version with --interpreter <interpreter name: node, python, python3, python3.

How do I run a Python script in a virtual environment?

To use the virtual environment you created to run Python scripts, simply invoke Python from the command line in the context where you activated it. For instance, to run a script, just run python myscript.py .

Can Nodejs run Python script?

The python-shell module by extrabacon is a simple way to run Python scripts from Node. js with basic, but efficient inter-process communication and better error handling.

How do I run a Python script in virtual environment Windows?

In "Program/script" textbox you set the path to Python executable (in my case is inside the virtualenv folder). "Add arguments" => Just the name of your Python script (name. ppy). "Start in" => The full path of your Python script (without the name.py).


1 Answers

After looking around a bit more, the question that I referenced at the top of the email had a clue in one of the answers, but not the answer.

When files end in '.py', pm2 calls 'python'... no matter what. I believe that there is a configuration file in pm2 that you could modify to change this behavior. I simply removed the '.py' from my script and specified the interpreter:

pm2 start ./strain_to_db --interpreter ./py3env/bin/python

Works perfectly. When I use pm2 to create a startup script, I will use absolute paths. Thanks for anyone who was looking, and I hope this helps someone in the future.

like image 62
slightlynybbled Avatar answered Sep 16 '22 11:09

slightlynybbled