How do I (safely) send a Python object to my Flask API?

Question

I am currently trying to build a Flask Web API that is able to receive a python object in a POST-request.

I am using Python 3.7.1 for creating the request and Python 2.7 for running the API. The API is set up to run on my local machine. The object I am trying to send to my API is a RandomForestClassifier object from sklearn.ensemble, but this could be any of a wide variety of object types.

So far I have tried to json.dumps() my object, but this object is not JSON serializable. I have also tried to pickle.dumps() my object, but this caused an error when trying to load the object on the API side. Also, since the API will receive requests from anonymous users, I am worried about performing a pickle.loads() on a possibly malicious object.

Is this a grounded worry? And if so, what is the best way to send any python object in a POST request?

The script performing the POST request:

import requests
import pickle

url = "http://localhost:5000/flask-api-function"

# the object I want to send is the 'model' object
data = pickle.dumps(model)

r = requests.post(url,data=data)

The Flask API:

@app.route('/flask-api-function', methods=['POST'])
def flask_api_function():

  model = pickle.loads(request.get_data())

This setup actually causes an error when trying to decode the data with pickle:

Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/flask/app.py", line 2309, in __call__
return self.wsgi_app(environ, start_response)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/larssuanet/Documents/enjins/dscs/flask_api.py", line 39, in store_model
model = pickle.loads(request.get_data())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 886, in load_proto
raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3

Is there a nice and safe way to do is?

Answer 1

When you pickle the object in python 3, pass the protocol keyword and set it to 2. This will ensure that it will work on python 2.

import requests
import pickle

url = "http://localhost:5000/flask-api-function"

# the object I want to send is the 'model' object
data = pickle.dumps(model,protocol=2)

r = requests.post(url,data=data)

Generally I would try to find a way to serialize the object into JSON since pickle has major security risks