Returning a higher-kinded closure that captures a reference

Question

I'm trying to write a method on a struct that returns a closure. This closure should take a &[u8] with an arbitrary lifetime 'inner as an argument and return the same type, &'inner [u8]. To perform its function, the closure also needs a (shared) reference to a member of the struct &self. Here is my code:

#![warn(clippy::pedantic)]

// placeholder for a large type that I can't afford to clone
struct Opaque(usize);

struct MyStruct<'a>(Vec<&'a Opaque>);

impl<'a> MyStruct<'a> {
    pub fn get_func_for_index(
        &'a self,
        n: usize,
    ) -> Option<impl for<'inner> Fn(&'inner [u8]) -> &'inner [u8] + 'a> {
        // the reference to the struct member, captured by the closure
        let opaque: &'a Opaque = *self.0.get(n)?;

        Some(move |i: &[u8]| {
            // placeholder: do something with the input using the &Opaque
            &i[opaque.0..]
        })
    }
}

fn main() {
    let o1 = Opaque(1);
    let o5 = Opaque(5);
    let o7 = Opaque(7);

    let x = MyStruct(vec![&o1, &o5, &o7]);

    let drop_five = x.get_func_for_index(1 /*Opaque(5)*/).unwrap();

    let data: Vec<u8> = Vec::from(&b"testing"[..]);

    assert_eq!(drop_five(&data[..]), b"ng");
}

If I try to compile it with rustc 1.54.0 (a178d0322 2021-07-26), I get the following error:

error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
  --> /tmp/lifetimes.rs:16:14
   |
16 |             &i[opaque.0..]
   |              ^^^^^^^^^^^^^
   |
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the body at 14:14...
  --> /tmp/lifetimes.rs:14:14
   |
14 |           Some(move |i: &[u8]| {
   |  ______________^
15 | |             // placeholder: do something with the input using the &Opaque
16 | |             &i[opaque.0..]
17 | |         })
   | |_________^
note: ...so that reference does not outlive borrowed content
  --> /tmp/lifetimes.rs:16:14
   |
16 |             &i[opaque.0..]
   |              ^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 6:6...
  --> /tmp/lifetimes.rs:6:6
   |
6  | impl<'a> MyStruct<'a> {
   |      ^^
note: ...so that return value is valid for the call
  --> /tmp/lifetimes.rs:10:17
   |
10 |     ) -> Option<impl for<'inner> Fn(&'inner [u8]) -> &'inner [u8] + 'a> {
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: higher-ranked subtype error
  --> /tmp/lifetimes.rs:7:5
   |
7  | /     pub fn get_func_for_index(
8  | |         &'a self,
9  | |         n: usize,
10 | |     ) -> Option<impl for<'inner> Fn(&'inner [u8]) -> &'inner [u8] + 'a> {
   | |_______________________________________________________________________^

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0495`.

It's quite a mouthful and I don't really understand what it's trying to tell me. The first part (first, the lifetime...) makes sense to me, the returned slice must not outlive the closure argument. The second part (but, the lifetime...) however seems weird to me - the + 'a annotation in the method signature refers to the closure itself (because it captures &'a self.foo), not to the value the closure returns.

Is it possible to change the code to model this correctly in rust, or is this construct just not supported at this time?

Answer 1

The problem is here:

Some(move |i: &[u8]| {

Every & has a lifetime on it, explicit or not. What is the lifetime of &[u8]? Clearly it should be "a lifetime chosen by the caller of the closure" (that is, a higher-ranked lifetime). But when the compiler encounters a reference type with a free lifetime parameter, even in the argument list of a closure, it will not assume that the lifetime is higher-ranked. The error message you get about the "anonymous lifetime #1" is the compiler confusedly trying to make a non-higher-ranked lifetime work.

The compiler could, in theory, work "backwards" from the impl Fn in the return type and recognize that the type of the closure needs to have this higher ranked lifetime. It's not quite clever enough to do that in this case, but there is a way to convince it: use a local function with a bounded type parameter to constrain the type of the closure to exactly the bound you want.

pub fn get_func_for_index(
    &self, // note 1
    n: usize,
) -> Option<impl 'a + for<'inner> Fn(&'inner [u8]) -> &'inner [u8]> { // note 2
    // the reference to the struct member, captured by the closure
    let opaque: &'a Opaque = *self.0.get(n)?;

    // helper function to constrain the closure type
    fn check<F: Fn(&[u8]) -> &[u8]>(f: F) -> F { // note 3
        f
    }

    Some(check(move |i| {
        // placeholder: do something with the input using the &Opaque
        &i[opaque.0..]
    }))
}

Playground

Please note the following:

1. &'a self is too conservative for this function because 'a is the lifetime parameter of the reference self contains, not the lifetime for which self is borrowed. In general, you should almost never write &'a self or &'a mut self where 'a is a named lifetime from an outer scope.
2. I find the + 'a easy to miss at the end of a long trait, especially a Fn trait with a return type. I recommend fronting the lifetime (putting it first) in cases like this to make clear that it relates more to the impl than to the &'inner [u8]. This is a stylistic choice.
3. Fn(&[u8]) -> &[u8] is actually exactly the same as for<'inner> Fn(&'inner [u8]) -> &'inner [u8], because the elision rules for Fn traits are the same as for regular functions. Either way is fine; I find the shorter version easier to read.

Similar questions

• Expected bound lifetime parameter, found concrete lifetime [E0271]
• How to declare a higher-ranked lifetime for a closure argument?
• Why does this closure require inlining or `dyn`? What does `dyn` do here?