Why do lifetimes differ when a function is called in a closure vs. being called directly in Rust?

Question

In the following code example:

fn default_values() -> &'static [u32] {
    static VALUES: [u32; 3] = [1, 2, 3];
    &VALUES
}

fn main() {
    let values: [u32; 3] = [4, 5, 6];
    let optional_values: Option<&[u32]> = Some(&values);

    // this compiles and runs fine 
    let _v = optional_values.unwrap_or_else(|| default_values());

    // this fails to compile
    let _v = optional_values.unwrap_or_else(default_values);
}

the last statement fails to compile with:

error[E0597]: `values` does not live long enough
  --> src/main.rs:8:49
   |
8  |     let optional_values: Option<&[u32]> = Some(&values);
   |                                                 ^^^^^^ borrowed value does not live long enough
...
12 | }
   | - borrowed value only lives until here
   |
   = note: borrowed value must be valid for the static lifetime...

I'm wondering:

1. what's happening that causes the difference in behaviour between the last two statements
2. whether the first unwrap_or_else(|| default_values()) is the correct way of handling this, or whether there's a better pattern

Answer 1

This happens because default_values implements Fn() -> &'static [u32], but not for<'a> Fn() -> &'a [u32]. Traits are invariant, so you can't coerce "something that implements Fn() -> &'static [u32]" to "something that implements Fn() -> &'a [u32]" (for some 'a smaller than 'static), even though, logically speaking, default_values could satisfy both.

When it's called in a closure, default_values() returns a &'static [u32], but it can be coerced immediately to a &'a u32, making the closure itself able to implement Fn() -> &'a [u32] (where the &'a is determined by the compiler).

As for why adding as fn() -> &'static [u32] works, I assume the compiler can recognize that the function pointer type fn() -> &'static [u32] is capable of implementing Fn() -> &'a [u32] for any 'a. I'm not sure why it doesn't also do this for ordinary functions and closures; perhaps a future compiler version could be smart enough to allow the original code.

Another solution is to make the type of default_values one that can implement the Fn trait you need:

fn default_values<'a>() -> &'a [u32] {
    static VALUES: [u32; 3] = [1, 2, 3];
    &VALUES
}

Instead of saying "this is a function that returns a 'static reference", the signature here says "this is a function that can return a reference of any lifetime". We know that "a reference of any lifetime" has to be a 'static reference, but the compiler sees the signatures as different because this one has an additional degree of freedom. This change is sufficient to make your original example compile.

Answer 2

There is no difference between a closure and a direct function call: It is just a matter of type inference.

closure that compiles:

let _v = optional_values.unwrap_or_else(|| default_values());
let _v = optional_values.unwrap_or_else(|| -> & [u32] {default_values()});

closure that not compiles:

let _v = unwrap_or_else(optional_values, || -> &'static [u32] {default_values()});

function that compiles:

let _v = unwrap_or_else(optional_values, default_values as fn() -> &'static _);

function that not compiles:

let _v = unwrap_or_else(optional_values, default_values);

A litte bit of explanation

Consider this equivalent code:

fn default_values() -> &'static [u32] {
    static VALUES: [u32; 3] = [1, 2, 3];
    &VALUES
}

fn unwrap_or_else<T, F>(slf: Option<T>, f: F) -> T where
    F: FnOnce() -> T, {
        match slf {
            Some(t) => t,
            None => f()
        }
    }

the following snippet:

fn main() {
    let values: [u32; 3] = [4, 5, 6];
    let optional_values: Option<&[u32]> = Some(&values);

    let _v = unwrap_or_else(optional_values, || -> &'static [u32] {default_values});

    // the above throws the same error of:
    //let _v = unwrap_or_else(optional_values, default_values);
}

fails:

error[E0597]: `values` does not live long enough
  --> src/main.rs:18:48
   |
18 |     let optional_values: Option<&[u32]> = Some(&values);
   |                                                ^^^^^^^
   |                                                |
   |                                                borrowed value does not live long enough
   |                                                cast requires that `values` is borrowed for `'static`
...
27 | }
   | - `values` dropped here while still borrowed

Look from the monomorphization side: assuming that the compiler infers that T resolves to the concrete type &'static [u32], and supposing that the produced code is something like:

fn unwrap_or_else_u32_sl_fn_u32_sl(slf: Option<&'static [u32]>,
                                   f: fn() -> &'static [u32]) -> &'static [u32] {
    ...
}

then the above monomorphization explains the error:

slf value is optional_values: an Option<&'a [u32]> that does not live enough and clearly cannot be cast because it does not satisfies the 'static lifetime requirement.

If you write:

let _v = unwrap_or_else(optional_values, || default_values());

// the same, expliciting the return type:
let _v = unwrap_or_else(optional_values, || -> & [u32] {default_values()});

It compiles: now the lifetime of the return type is compatible with the optional_values lifetime.

Finally, I'm not able to explain why, but the evidence shows that the cast as fn() -> &'static _ helps the compiler to be sure that decoupling lifetimes bound to optional_values and default_values is safe.