Why does my trait need a lifetime parameter?

Question

Being a Rust newbie, I probably somewhat naively started with this:

...

pub trait Decode<T> {
    fn decode_from<R: io::Read + ?Sized>(&mut self, stream: &mut R) -> T;
}

pub struct MQTTFrame<'a> {
    pub payload: &'a Vec<u8>,
}

pub struct MQTTFrameDecoder<'a> {
    pub payload: &'a mut Vec<u8>,
}

impl<'a> Decode<MQTTFrame<'a>> for MQTTFrameDecoder<'a> {
    fn decode_from<R: io::Read + ?Sized>(&mut self, stream: &mut R) ->    MQTTFrame<'a> {
        stream.read(&mut self.payload);
        MQTTFrame{ payload: self.payload }
    }
}

Which, when trying to compile, was greeted with:

src/testbed/mod.rs:31:24: 31:36 error: cannot infer an appropriate   lifetime for automatic coercion due to conflicting requirements [E0495]
src/testbed/mod.rs:31         MQTTFrame{ payload: self.payload }
                                                  ^~~~~~~~~~~~
src/testbed/mod.rs:29:5: 32:6 help: consider using an explicit lifetime  parameter as shown: fn decode_from<R: io::Read + ?Sized>(&'a mut self,   stream: &mut R)
 -> MQTTFrame<'a>
src/testbed/mod.rs:29     fn decode_from<R: io::Read + ?Sized>(&mut self, stream: &mut R) -> MQTTFrame<'a> {
src/testbed/mod.rs:30         stream.read(&mut self.payload);
src/testbed/mod.rs:31         MQTTFrame{ payload: self.payload }
src/testbed/mod.rs:32     }

Somewhere on StackOverflow - sorry, I forgot where - someone in a similar case suggested to add a lifetime parameter like so (omitting unchanged code):

pub trait Decode<'a, T> {
    fn decode_from<R: io::Read + ?Sized>(&'a mut self, stream: &mut R) -> T;
}

impl<'a> Decode<'a, MQTTFrame<'a>> for MQTTFrameDecoder<'a> {
    fn decode_from<R: io::Read + ?Sized>(&'a mut self, stream: &mut R) -> MQTTFrame<'a> {
        stream.read(&mut self.payload);
        MQTTFrame{ payload: self.payload }
    }
}

And lo and behold! It compiles. Now if I could only understand why it compiles. Could someone explain

1. why the original code did not compile?
2. why the modified code compiles?

Answer 1

Here's a reduced testcase that fails to compile (playpen):

pub trait Decode<T> {
    fn decode_from<'b>(&'b mut self) -> T;
}

pub struct MQTTFrame<'a> {
    pub payload: &'a Vec<u8>,
}

pub struct MQTTFrameDecoder<'a> {
    pub payload: &'a mut Vec<u8>,
}

impl<'a> Decode<MQTTFrame<'a>> for MQTTFrameDecoder<'a> {
    fn decode_from<'b>(&'b mut self) -> MQTTFrame<'a> {
        MQTTFrame{ payload: self.payload }
    }
}

Note that I have elided the lifetimes for the decode_from function and removed the redundant stream parameter.

It is clear then that the function is taking a reference with an arbitrarily short lifetime 'b, and then extending it to have lifetime 'a. This is a problem with mutable references as then you can borrow something mutably and immutably at the same time:

fn main() {
    let mut v = vec![];
    /* lifetime 'a */ {
        let mut decoder = MQTTFrameDecoder{ payload: &mut v };
        let frame: MQTTFrame;
        /* lifetime 'b */ {
            frame = decoder.decode_from(); // borrows decoder just for lifetime 'b
        }
        // v is mutably borrowed (by decoder) and immutably borrowed (by frame) at the same time! oops!
        decoder.payload.push(1);
        println!("{:?}", frame.payload);
    }
}

For this reason the borrow checker refuses to let the function compile.

If you force the reference to decoder to have lifetime 'a, though, then there is no longer a problem. The compiler cannot use the reference with the shorter lifetime, it must mutably borrow decoder for longer, and so the compiler should give us an error when we try to borrow it again.

In order to achieve this, we would like to write

fn decode_from(&'a mut self) -> MQTTFrame<'a> {
    MQTTFrame{ payload: self.payload }
}

But now we get an error:

<anon>:14:5: 16:6 error: method `decode_from` has an incompatible type for trait:
 expected bound lifetime parameter 'b,
    found concrete lifetime [E0053]

To fix this, we need to have our trait be aware that you can only decode_from certain lifetimes, not arbitrary ones. So change decode to

pub trait Decode<'a, T> {
    fn decode_from(&'a mut self) -> T;
}

And make the appropriate change to the implementation

impl<'a> Decode<'a, MQTTFrame<'a>> for MQTTFrameDecoder<'a> { ... }

Now if we try the code above (playpen is.gd/BLStYq), the borrow checker complains:

<anon>:28:9: 28:24 error: cannot borrow `*decoder.payload` as mutable more than once at a time [E0499]
<anon>:28         decoder.payload.push(1);

That's because, now, the reference to decoder must have lifetime 'a when it is taken in order to call the function decode_from. Comment out the offending line and the rest of the example compiles! This code is now safe because no mutable lifetimes are being extended.

---

Aside:

As the reference to decoder must live as long as the decoder itself, you can't actually use decoder at all after you have called decode_from. As this is the case, it may be better to express this by taking self instead of &'a mut self. Then the syntax is a little cleaner, and it is obvious that once a decoder has been used then it can't be used again.

pub trait Decode<T> {
    fn decode_from(self) -> T;
}

pub struct MQTTFrame<'a> {
    pub payload: &'a Vec<u8>,
}

pub struct MQTTFrameDecoder<'a> {
    pub payload: &'a mut Vec<u8>,
}

impl<'a> Decode<MQTTFrame<'a>> for MQTTFrameDecoder<'a> {
    fn decode_from(self) -> MQTTFrame<'a> {
        MQTTFrame{ payload: self.payload }
    }
}

Answer 2

Lifetime elision works only in very simple cases. This makes them weak, but easy to explain (also simple cases are surprisingly common).

As soon as you have a generic lifetime parameter, elision no longer applies – the compiler refuses to guess your intention.