Using Sled, how do I serialize and deserialize?

Question

I'm playing around with the crate sled and attempting a simple serialization and deserialization exercise using bincode just to get a handle on usage.

While I can get insertion to work, attempting to get a range of results seems more difficult. Here I attempt to put in two records: a key of 42 with the value of "Alice" and a key of 69 with a value of "Bob and then retrieve and print them. I'm having a hard time reconciling deserialization of the vectors that come out:

use crate::db::Database;
use sled::Db;
use bincode;


pub struct SledDatabase {
    db: Db,
}

impl Database for SledDatabase {
    fn create(&self) {
        // Not handling errors; just an example.
        let key: i64 = 42;
        println!("ser {:?}", bincode::serialize(&key).unwrap());
        self.db.insert(bincode::serialize(&key).unwrap(), bincode::serialize("Alice").unwrap());
        let key2: i64 = 69;
        self.db.insert(bincode::serialize(&key2).unwrap(), bincode::serialize("Bob").unwrap());
    }

    fn query(&self, value : i64) {
        let range = value.to_ne_bytes();
        let mut iter = self.db.range(range..);
        while let Some(item) = iter.next() {
            let (k, v) = item.unwrap();
            println!("res {:?}", k);
            let key: i64 = bincode::deserialize(&k).unwrap();
            let value: String = bincode::deserialize(&v).unwrap();
            println!("age = {}", key);
            println!("name = {}", value);
        }
    }
}

impl SledDatabase {
    pub fn connect() -> SledDatabase {
        // use sled::{ConfigBuilder, Error};
        // let config = ConfigBuilder::new().temporary(true).build();
        SledDatabase { db: Db::open("sled.db").unwrap() }
    }
}

Attempting to interact with this using a console entry listener I've pieced together, I get the following output:

ser [42, 0, 0, 0, 0, 0, 0, 0]
>> 1
res [42]
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Custom { kind: UnexpectedEof, error: "failed to fill whole buffer" })', src/libcore/result.rs:1084:5

This occurs on either attempt to deserialize (either the String or the i64 value. Interestingly, the output seems to indicate that either sled or bincode are truncating the key value.

The examples of sled usage don't seem to address retrieving values, but looking through some documentation and parts of the source, I get the sense that serde serialization is a valid way of getting things in and out of sled.

Answer 1

I cannot reproduce this issue with the code given above, I'm unsure if the sled implementation has perhaps changed a size of something related to the range function, and now match the expectations of the code, or if this was just a bug.

None the less, I ran into this error message due to some carelessness on my part.

fn main() -> Result<(), Error> {
    let db = sled::open("foo.sled_db").unwrap();
    let codec = bincode::config();

    db.insert("uhoh", codec.serialize(&0)?)?;
    let ser = &db.get("uhoh")?.unwrap();
    let de1: i32 = codec.deserialize(ser)?;
    println!("ok... {}", de1);
    let de2: usize = codec.deserialize(ser)?;
    println!("not ok {}", de2);
}

• I've omitted implementations of From<sled::Error> and From<bincode::Error> for Error, to simplify things.

ok... 0

Error: Bincode(Io(Custom { kind: UnexpectedEof, error: "failed to fill whole buffer" })

The thing to note here is that (as you say) value which was serialized is smaller, than the type we attempt to deserialize into.

println!(
       "{} {}",
       core::mem::size_of::<usize>(),
       core::mem::size_of_val(&0)
);

println!(
        "{} {}",
        core::mem::size_of::<usize>(),
        core::mem::size_of::<i32>()
);

8 4

8 4

To fix this I serialized this with the value &0usize instead of &0. Using serde serialization for sled keys and values indeed does seem fine, but you do need to be careful about the inferred size of values.