"borrowed data escapes outside of closure" only when using &mut or threads?

Question

When trying to reassign a reference to point somewhere else from inside a closure, I noticed a strange behavior that I cannot explain, shown by this minimal example:

fn main() {
    let mut foo: i32 = 5;
    let mut foo2: i32 = 6;
    let mut borrower = &mut foo; // compiles OK without mut here and below
    let mut c = || {
        borrower = &mut foo2;    // compiles OK without mut here and above
    };
}

this yields the following error ONLY when the references are &mut:

error[E0521]: borrowed data escapes outside of closure
  --> src/main.rs:25:9
   |
23 |     let mut borrower = &mut foo;
   |         ------------ `borrower` declared here, outside of the closure body
24 |     let mut c = || {
25 |         borrower = &mut foo2;
   |         ^^^^^^^^^^^^^^^^^^^^

What does this error actually mean here? Why would it be unsafe to do this, given that it's clear that the closure is only alive while foo2 is still alive? Why does it matter whether it's a &mut reference or not?

When trying the same from a a scoped thread, it NEVER compiles, with or without mut:

fn main() {
    let mut foo: i32 = 5;
    let mut foo2: i32 = 6;
    let a = Arc::new(Mutex::new(&mut foo)); // removing mut does NOT fix it
    println!("{}", a.lock().unwrap());
    
    thread::scope(|s| {
        let aa = a.clone();
        s.spawn(move ||{
            *aa.lock().unwrap() = &mut foo2; // removing mut does NOT fix it
        });
    });
}

When mut is removed, the program compiles without error. Why is the behavior different here from the first example, where removing mut satisfies the compiler?

My research has lead me to believe that it might have something to do with the FnOnce, FnMut and Fn Traits of closures, but I am stuck.

Answer 1

Consider the following code:

fn main() {
    let mut foo: i32 = 5;
    let mut foo2: i32 = 6;
    let mut borrower = &mut foo;
    let mut called = false;
    let mut c = || {
        if !called {
            borrower = &mut foo2;
            called = true;
        } else {
            foo2 = 123;
        }
    };
    c();
    c();
    *borrower = 456;
}

If the compiler would look at the code like you wanted it to, this code would be valid: we don't borrow foo2 in the current branch, so it is not borrowed. But this code is clearly not: we mutate foo2 while it is borrowed, from the previous closure call.

If you'll ask how the compiler figures this out, then we'll need to look at how the desugared closure looks like.

Closures desugars to structs that implement the Fn family of traits. Here's how, roughly, our closure desugars:

struct Closure<'borrower, 'foo2> {
    borrower: &'borrower mut &'foo2 mut i32,
    foo2: &'foo2 mut i32,
}

// Forward `FnOnce` to `FnMut`. This is not really relevant for us, and I left it only for completeness.
impl FnOnce<()> for Closure<'_, '_> {
    type Output = ();
    extern "rust-call" fn call_once(mut self, (): ()) -> Self::Output {
        self.call_mut(())
    }
}

impl<'borrower, 'foo2> FnMut<()> for Closure<'borrower, 'foo2> {
    extern "rust-call" fn call_mut<'this>(&'this mut self, (): ()) -> Self::Output {
        *self.borrower = self.foo2;
    }
}

// let mut c = || {
//     borrower = &mut foo2;
// };
let mut c = Closure {
    borrower: &mut borrower,
    foo2: &mut foo2,
}

See the problem? We're trying to assign self.foo2 to *self.borrower, but we cannot move out of self.foo as we only have a mutable reference to self. We can mutably borrow it, but only for the lifetime of self - 'this, and it is not enough. We need the full lifetime foo2.

However, when the references are immutable, we don't need to move out of self.foo2 - we can just copy it. This creates a reference with the desired lifetime, because immutable references are Copy.

The reason it wants in the code I brought in a comment, without Mutex (without move, I hope it is obvious why it doesn't work with move), is that spawn() takes FnOnce, so the compiler knows we cannot call the closure twice. Technically, We have self and not &mut self, so we can move out of its fields.

It works if we force FnOnce too:

fn force_fnonce(f: impl FnOnce()) {}
force_fnonce(|| {
    borrower = &mut foo2;
});

The reason it does not work with your scoped threads snippet, even though it requires FnOnce, is completely different: it's again because of the move. Because of it, foo2 is local to the closure, and borrowing it yields a reference that is only valid in the closure, as it is destroyed when the closure exits. Fixing it requires borrowing foo2 instead of moving it. We cannot get rid of the move because of aa, so we need to partially move closure captures. The way to do that is as follows:

fn main() {
    let mut foo: i32 = 5;
    let mut foo2: i32 = 6;
    let a = Arc::new(Mutex::new(&mut foo));
    println!("{}", a.lock().unwrap());

    thread::scope(|s| {
        let aa = a.clone();
        let foo2_ref = &mut foo2;
        s.spawn(move || {
            *aa.lock().unwrap() = foo2_ref;
        });
    });
}

And this code indeed compiles, even with the &mut.