Quick and resilient way to expose a python library API over the network as well as locally

Question

I'm looking for an easy and dependency-light way of wrapping a python library to expose it over:

a) The network, either via HTTP or some other custom protocol, it doesn't matter that much, and encryption isn't required. b) The local machine, the main purpose here is to avoid library import overhead, ideally, this would happen via an efficient mechanism ala pipes or shared memory to minimize the number of data copies and [de]serialization.

It seems like it's an easy enough job to just create a class that runs constantly, with e.g. an HTTP interface mirroring the library functionality and returning e.g. pickled objects corresponding to the answers. But getting it to work efficiently and cover various edge-cases seems tedious and I'm wondering if there's a better way to do this, ideally one that's built into python itself.

Ray seems to have some functionality for this using Actors, but it seems rather heavy-weight and prone to fail when being installed, so I'm curios what alternatives exist.

Also, might be too much of a "library question", if you think it's better suited for another stack exchange website please tell me which and I'll remove it from here.

Answer 1

Your best options are Flask or FASTAPi. Both are lightweight and very resilient web frameworks, that are also pretty easy to start working with (it may be a matter of adding one decorator to your function to achieve your goal). You may also couple your API with Swagger UI, to have visual interaction with your API resources.

P.S. Ray actors has nothing to do with your request (they are simply statefull objects meant to be run/used in a distributed fashion).

Answer 2

A relatively simple and lightweight solution is to use RPC and expose your existing functions.

For example:

server:

from xmlrpc.server import SimpleXMLRPCServer

def is_even(n):
    return n % 2 == 0

server = SimpleXMLRPCServer(("localhost", 8000))
print("Listening on port 8000...")
server.register_function(is_even, "is_even")
server.serve_forever()

Functions can also be registered in a decorator way instead of calling server_function(is_even, "is_even"):

@server.register_function
def is_even(n):
    return n % 2 == 0

client:

import xmlrpc.client

with xmlrpc.client.ServerProxy("http://localhost:8000/") as proxy:
    print("3 is even: %s" % str(proxy.is_even(3)))
    print("100 is even: %s" % str(proxy.is_even(100)))

Further information: https://docs.python.org/3/library/xmlrpc.server.html#