How can I constrain the lifetime of a struct to that of a 'parent' struct?

Question

I am using the FFI to write some Rust code against a C API with strong notions of ownership (the libnotmuch API, if that matters).

The main entry point to the API is a Database; I can create Query objects from the Database. It provides destructor functions for databases and queries (and a lot of other objects).

However, a Query cannot outlive the Database from which it was created. The database destructor function will destroy any undestroyed queries, etc., after which the query destructors don't work.

So far, I have the basic pieces working - I can create databases and queries, and do operations on them. But I am having difficulty encoding the lifetime bounds.

I'm trying to do something like this:

struct Db<'a>(...) // newtype wrapping an opaque DB pointer
struct Query<'a>(...) // newtype wrapping an opaque query pointer

I have implementations of Drop for each of these that call the underlying C destructor functions.

And then have a function that creates a query:

pub fun create_query<?>(db: &Db<?>, query_string: &str) -> Query<?>

I do not know what to put in place of the ?s so that the Query returned is not allowed to outlive the Db.

How can I model the lifetime constraints for this API?

Answer 1

When you want to bind the lifetime of an input parameter to the lifetime of the return value, you need to define a lifetime parameter on your function and reference it in the types of your input parameter and return value. You can give any name you want to this lifetime parameter; often, when there are few parameters, we just name them 'a, 'b, 'c, etc.

Your Db type takes a lifetime parameter, but it shouldn't: a Db doesn't reference an existing object, so it has no lifetime constraints.

To correctly force the Db to outlive the Query, we must write 'a on the borrowed pointer, rather than on the lifetime parameter on Db that we just removed.

pub fn create_query<'a>(db: &'a Db, query_string: &str) -> Query<'a>

However, that's not enough. If your newtypes don't reference their 'a parameter at all, you'll find that a Query can actually outlive a Db:

Editor's note: This code no longer compiles since Rust 1.0. You must use 'a in some way in the body of Query.

struct Db(*mut ());
struct Query<'a>(*mut ());  // '

fn create_query<'a>(db: &'a Db, query_string: &str) -> Query<'a> {  // '
    Query(0 as *mut ())
}

fn main() {
    let query;
    {
        let db = Db(0 as *mut ());
        let q = create_query(&db, "");
        query = q; // shouldn't compile!
    }
}

That's because, before Rust 1.0, lifetime parameters are bivariant, i.e. the compiler may substitute the parameter with a longer or a shorter lifetime in order to meet the caller's requirements.

When you store a borrowed pointer in a struct, the lifetime parameter is treated as covariant: that means the compiler may substitute the parameter with a shorter lifetime, but not with a longer lifetime.

We can ask the compiler to treat your lifetime parameter as covariant manually by adding a PhantomData marker to our struct:

use std::marker::PhantomData;

struct Db(*mut ());
struct Query<'a>(*mut (), PhantomData<&'a ()>);

fn create_query<'a>(db: &'a Db, query_string: &str) -> Query<'a> {    // '
    Query(0 as *mut (), PhantomData)
}

fn main() {
    let query;
    {
        let db = Db(0 as *mut ());
        let q = create_query(&db, ""); // error: `db` does not live long enough
        query = q;
    }
}

Now, the compiler correctly rejects the assignment to query, which outlives db.

---

Bonus: If we change create_query to be a method of Db, rather than a free function, we can take advantage of the compiler's lifetime inference rules and not write 'a at all on create_query:

use std::marker::PhantomData;

struct Db(*mut ());
struct Query<'a>(*mut (), PhantomData<&'a ()>);

impl Db {
    //fn create_query<'a>(&'a self, query_string: &str) -> Query<'a>
    fn create_query(&self, query_string: &str) -> Query {
        Query(0 as *mut (), PhantomData)
    }
}

fn main() {
    let query;
    {
        let db = Db(0 as *mut ());
        let q = db.create_query(""); // error: `db` does not live long enough
        query = q;
    }
}

When a method has a self parameter, the compiler will prefer linking the lifetime of that parameter with the result, even if there are other parameters with lifetimes. For free functions though, inference is only possible if only one parameter has a lifetime. Here, because of the query_string parameter, which is of type &'a str, there are 2 parameters with a lifetime, so the compiler cannot infer which parameter we want to link the result with.