Cannot infer an appropriate lifetime for a closure that returns a reference

Question

Considering the following code:

fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {     Box::new(move || &t) } 

What I expect:

• The type T has lifetime 'a.
• The value t live as long as T.
• t moves to the closure, so the closure live as long as t
• The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
• There is no lifetime problem, the code compiles.

What actually happens:

• The code does not compile:

error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements  --> src/lib.rs:2:22   | 2 |     Box::new(move || &t)   |                      ^^   | note: first, the lifetime cannot outlive the lifetime  as defined on the body at 2:14...  --> src/lib.rs:2:14   | 2 |     Box::new(move || &t)   |              ^^^^^^^^^^ note: ...so that closure can access `t`  --> src/lib.rs:2:22   | 2 |     Box::new(move || &t)   |                      ^^ note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...  --> src/lib.rs:1:8   | 1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {   |        ^^   = note: ...so that the expression is assignable:           expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>              found std::boxed::Box<dyn std::ops::Fn() -> &T> 

I do not understand the conflict. How can I fix it?

Answer 1

Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.

tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).

---

Implementing it manually

As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.

What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.

struct Baz<T>(T);  impl<T> Baz<T> {     fn call(&self) -> &T {         &self.0     } }  fn make_baz<T>(t: T) -> Box<Baz<T>> {     Box::new(Baz(t)) } 

This is pretty equivalent to your boxed closure. Let's try to use it:

let outside = {     let s = "hi".to_string();     let baz = make_baz(s);     println!("{}", baz.call()); // works      baz };  println!("{}", outside.call()); // works too 

This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.

It gets more interesting when we add a single character:

let outside = {     let s = "hi".to_string();     let baz = make_baz(&s);  // <-- NOW BORROWED!     println!("{}", baz.call()); // works      baz };  println!("{}", outside.call()); // doesn't work! 

Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.

The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.

Writing a trait for it

So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:

trait MyFn {     type Output;     fn call(&self) -> Self::Output; } 

In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.

Let's implement it!

impl<T> MyFn for Baz<T> {     type Output = ???;     fn call(&self) -> Self::Output {         &self.0     } } 

Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?

Let's check the function we implemented before:

impl<T> Baz<T> {     fn call(&self) -> &T {         &self.0     } } 

So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:

fn call<'s>(&'s self) -> &'s T 

Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).

(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)

But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.

What about the future?

Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).

Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:

trait MyFn {     type Output<'a>;   // <-- we added <'a> to make it generic     fn call(&self) -> Self::Output; } 

The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:

fn call<'s>(&'s self) -> Self::Output<'s> 

So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:

impl<T> MyFn for Baz<T> {     type Output<'a> = &'a T;     fn call(&self) -> Self::Output {         &self.0     } } 

To return a boxed MyFn we would need to write this (according to this section of the RFC:

fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {     Box::new(Baz(t)) } 

And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)

---

Summary

What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.

With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.