How can I create a language independent library using Python?

Question

If I create a package in Python, then another Python user can import that package and interface with it.

How can I create a package so that it doesn't matter what language the other user is calling the library with?

I could specify input and output file formats so that another language can interface with my Python code by merely providing the input files and reading the output files. However, creating the input and output files is very expensive computationally. Is there an easier solution?

Answer 1

In the general case, two different languages can't live in the same process. So you have to make one language call another throught interprocess communication (IPC).

The simplest and usually effective way to do so is via input/output of the calee "library". It usually has some kind of serialisation overhead, but in a typical matlab/python interaction it should be not noticeable.

In this configuration the slowest part is the startup of the python process which can be supressed by keeping the same process alive between two call.

Here an example of ipc between a main program in python (but could be written in any language) and a library in python using stdin/stdout and json as serialization

---

#main program in foreign language

import mylibwrapper

print(mylibwrapper.call("add",[1,2]))

mylibwrapper.exit()

---

#mylibwrapper.py supposed written in foreign language
import subprocess
import json

process = subprocess.Popen([
    "python3",
     "mylib.py"],
    stdin = subprocess.PIPE,
    stdout = subprocess.PIPE,
    encoding = "utf8")

def call(name,args):
  process.stdin.write(json.dumps([name,args]))
  process.stdin.write("\n")
  process.stdin.flush()
  result = process.stdout.readline()
  return(result)
def exit():
  process.terminate()

---

#mylib.py

import json, sys

def add(arg1,arg2):
  return arg1 + arg2

if __name__ == "__main__":

  for line in sys.stdin:
    name, args = json.loads(line)
    function = { "add" : add }[name]
    sys.stdout.write(json.dumps(function( *args)))
    sys.stdout.write("\n")
    sys.stdout.flush()

Answer 2

You can use Cython to relatively easily extend your python code so that it can be compiled as a C library. Mostly, this just involved replacing the def keyword with cdef public for the functions you want to expose, and annotating your variables types.

Here's an example of such Cython code:

cdef public void grail(int num):
    printf("Ready the holy hand grenade and count to %i", num) 

At that point, many languages have Foreign Function Interfaces (FFI) to C code.