How can I implement serde::Deserialize for arrays larger than 32? [duplicate]

Question

I need to use 33-byte arrays. It looks like arrays up to and including 32 elements serialise without problems, but 33-byte arrays cause:

error[E0277]: the trait bound `[u8; 33]: _::_serde::Deserialize<'_>` is not satisfied
  --> src/main.rs:54:2
   |
54 |     foo: [u8; 33]
   |     ^^^ the trait `_::_serde::Deserialize<'_>` is not implemented for `[u8; 33]`
   |
   = help: the following implementations were found:
             <&'a [u8] as _::_serde::Deserialize<'de>>
             <[T; 0] as _::_serde::Deserialize<'de>>
             <[T; 10] as _::_serde::Deserialize<'de>>
             <[T; 11] as _::_serde::Deserialize<'de>>
           and 30 others
   = note: required by `_::_serde::de::MapAccess::next_value`

They evidently decided that 32 was a good place to stop. How can I add support for 33 element arrays?

Answer 1

You can't implement a trait you don't own for a type you don't own, but unfortunately, in this particular case, you don't have to.

---

For serialization, support of big arrays is pretty easy

struct S {
   #[serde(serialize_with = "<[_]>::serialize")]
   arr: [u8; 256],
}

Source

This coerces the array to a slice and serializes that instead.

---

For de-serialization, it is a bit more complicated.

serde's own author recommends the following

#[macro_use] extern crate serde_derive;

extern crate serde; extern crate serde_json;

use std::fmt; use std::marker::PhantomData;
use serde::ser::{Serialize, Serializer, SerializeTuple};
use serde::de::{Deserialize, Deserializer, Visitor, SeqAccess, Error};

trait BigArray<'de>: Sized {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
        where S: Serializer;
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
        where D: Deserializer<'de>;
}

macro_rules! big_array {
    ($($len:expr,)+) => {
        $(
            impl<'de, T> BigArray<'de> for [T; $len]
                where T: Default + Copy + Serialize + Deserialize<'de>
            {
                fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
                    where S: Serializer
                {
                    let mut seq = serializer.serialize_tuple(self.len())?;
                    for elem in &self[..] {
                        seq.serialize_element(elem)?;
                    }
                    seq.end()
                }

                fn deserialize<D>(deserializer: D) -> Result<[T; $len], D::Error>
                    where D: Deserializer<'de>
                {
                    struct ArrayVisitor<T> {
                        element: PhantomData<T>,
                    }

                    impl<'de, T> Visitor<'de> for ArrayVisitor<T>
                        where T: Default + Copy + Deserialize<'de>
                    {
                        type Value = [T; $len];

                        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                            formatter.write_str(concat!("an array of length ", $len))
                        }

                        fn visit_seq<A>(self, mut seq: A) -> Result<[T; $len], A::Error>
                            where A: SeqAccess<'de>
                        {
                            let mut arr = [T::default(); $len];
                            for i in 0..$len {
                                arr[i] = seq.next_element()?
                                    .ok_or_else(|| Error::invalid_length(i, &self))?;
                            }
                            Ok(arr)
                        }
                    }

                    let visitor = ArrayVisitor { element: PhantomData };
                    deserializer.deserialize_tuple($len, visitor)
                }
            }
        )+
    }
}

big_array! {
    40, 48, 50, 56, 64, 72, 96, 100, 128, 160, 192, 200, 224, 256, 384, 512,
    768, 1024, 2048, 4096, 8192, 16384, 32768, 65536, }

#[derive(Serialize, Deserialize)]
struct S {
    #[serde(with = "BigArray")]
    arr: [u8; 64], }

fn main() {
    let s = S { arr: [1; 64] };
    let j = serde_json::to_string(&s).unwrap();
    println!("{}", j);
    serde_json::from_str::<S>(&j).unwrap(); }

Source

This has been published in the serde-big-array crate.