How to convince the compiler that my async `FnMut` is not invoked in parallel?

Question

In one crate, I have a general-purpose wrapper for temporary files:

pub struct TempFile {
    // ...
}

In another crate, I have a general-purpose async retrying wrapper:

pub async fn retry<F, R>(mut func: F) -> Result<(), String>
where
    F: FnMut() -> R,
    R: Future<Output = Result<(), String>>
{
    for _ in 0..5 {
        if func().await.is_ok() {
            return Ok(());
        }
    }
    Err("failed".to_owned())
}

In third crate, I have a general-purpose download function that downloads into a temporary file:

async fn download(output: &mut TempFile) -> Result<(), String> {
    // ...
}

Now I want to add a retrying download function like this:

async fn download_with_retry(output: &mut TempFile) -> Result<(), String> {
    retry(|| { download(output) }).await
}

However, this does not compile:

error: captured variable cannot escape `FnMut` closure body
  --> src/lib.rs:16:20
   |
15 |     async fn download_with_retry(output: &mut TempFile) -> Result<(), String> {
   |                                  ------ variable defined here
16 |         retry(|| { download(output) }).await
   |                -   ^^^^^^^^^------^
   |                |   |        |
   |                |   |        variable captured here
   |                |   returns an `async` block that contains a reference to a captured variable, which then escapes the closure body
   |                inferred to be a `FnMut` closure
   |
   = note: `FnMut` closures only have access to their captured variables while they are executing...
   = note: ...therefore, they cannot allow references to captured variables to escape

I understand why this is happening: even though the retry function always waits for func to finish before invoking it again, the compiler cannot see this, and has to assume that multiple parallel invocations are possible, leading to multiple simultaneous mutable borrows of output.

I know I can work around this by passing Rc<RefCell<TempFile>> to download_with_retry, but that means it leaks into that function's public API for no good reason.

Is there another solution? Ideally by modifying only the implementation of retry and/or download_with_retry, while keeping their signatures the same?

Playground link

Answer 1

Ideally by modifying only the implementation of retry and/or download_with_retry, while keeping their signatures the same?

The signature cannot stay the same. The problem is hinted at in the error: if a closure captures &mut, you can't use that &mut in a Future it returns, except by moving the &mut out of the captures (making the function a FnOnce, or using Option::take() or something like that). In general, Fns and FnMuts cannot “lend”: return things that borrow from themselves.

Rust 1.85 added async function traits and async closures to allow this. To use them, change the signature of retry() to

pub async fn retry<F>(mut func: F) -> Result<(), String>
where
    F: AsyncFnMut() -> Result<(), String>
{

and the usage to

retry(async || download(output).await).await

However, there is currently missing functionality around being able to express Send bounds on async function futures, so you may not be able to actually use this. Hopefully the relevant features will be implemented and stabilized soon; return type notation is part of the solution.