Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Sklearn Model (Python) with NodeJS (Express): how to connect both?

I have a web server using NodeJS - Express and I have a Scikit-Learn (machine learning) model pickled (dumped) in the same machine.

What I need is to demonstrate the model by sending/receiving data from it to the server. I want to load the model on startup of the web server and keep "listening" for data inputs. When receive data, executes a prediction and send it back.

I am relatively new to Python. From what I've seen I could use a "Child Process" to execute that. I also saw some modules that run Python script from Node.

The problem is I want to load the model once and let it be for as long as the server is on. I don't want to keep loading the model every time due to it's size. How is the best way to perform that?

The idea is running everything in a AWS machine.

Thank you in advance.

like image 622
Jean Phelippe Avatar asked Mar 05 '17 16:03

Jean Phelippe


People also ask

Can Python and NodeJS work together?

js and Python finding out that Node. js is awesome for Web Development and Python for Data Sciences. Actually, we don't need to always stick with the same programming language as there are ways to use them both together. In this article, I will show you an example of how to use a Python script from the Node.


1 Answers

My recommendation: write a simple python web service (personally recommend flask) and deploy your ML model. Then you can easily send requests to your python web service from your node back-end. You wouldn't have a problem with the initial model loading. it is done once in the app startup, and then you're good to go

DO NOT GO FOR SCRIPT EXECUTIONS AND CHILD PROCESSES!!! I just wrote it in bold-italic all caps so to be sure you wouldn't do that. Believe me... it potentially go very very south, with all that zombie processes upon job termination and other stuff. let's just simply say it's not the standard way to do that.

You need to think about multi-request handling. I think flask now has it by default

I am just giving you general hints because your problem has been generally introduced.

like image 154
Alireza Avatar answered Sep 22 '22 17:09

Alireza