How to call Rust async method from Python?

Question

I want to use a Rust async method in Python. I'm trying to use PyO3 or rust-cpython.

For example, for sync Rust functions, I can use,

#[pyfunction]
fn myfunc(a: String) -> PyResult<String> {
   let mut contents = String::new();
   contents = a.to_string() + " appended";
   Ok((contents))
}

#[pymodule]
fn MyModule(py: Python, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pyfunction!(urlshot))?;
    Ok(())
}

For async methods, how can I do it? For example, I want to call the following method in Python,

async fn hello_world() {
    println!("hello, world!");
}

Answer 1

Since there was no easy way of solving this issue (at least, I hadn't found), I converted my async method to sync one. And called it on Python side as,

async fn my_method(s: &str) -> Result<String, Error> {
    // do something
}

#[pyfunction]
fn my_sync_method(s: String) -> PyResult<String> {
    let mut rt = tokio::runtime::Runtime::new().unwrap();
    let mut contents = String::new();
    rt.block_on(async {
        result = format!("{}", my_sync_method(&s).await.unwrap()).to_string();
    });
   Ok((result))
}

#[pymodule]
fn MyModule(py: Python, m: &PyModule) -> PyResult<()> {
    m.add_wrapped(wrap_pyfunction!(my_sync_method))?;
    Ok(())
}

Edited

In the Cargo.toml file, I added the following dependencies,

[dependencies.pyo3]
git = "https://github.com/PyO3/pyo3"
features = ["extension-module"]

After running cargo build --release, target/release/libMyModule.so binary file is generated. Rename it as MyModule.so and it now can be imported from Python.

import MyModule
result = MyModule.my_sync_method("hello")

Using setuptools-rust, I could bundle it as an ordinary Python package.

All of the above code and commands are tested on newly-released Linux Mint 20. On MacOS, the binary file will be libMyModule.dylib.

Answer 2

If you want to use Python to control Rust's async function, I don't think it will work (Or at least it is very complicated, as you need to connect two different future mechanism). For async functions, Rust compiler will maintain a state machine to manage the coroutines run correctly under await's control. This is an internal state of Rust applications and Python cannot touch it. Similarly Python interpreter also has a state machine that cannot be touched by Rust.

I do found this topic about how to export an async function using FFI. The main idea is to wrap the async in a BoxFuture and let C control the timing of returning it to Rust. However, you cannot use BoxFuture in PyO3 since its pyfunction macro cannot convert a function returns BoxFuture to a Python callback. You may try to create a library using FFI and use python's cffi module to load it.