Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using serde for two (de)serialization formats

Tags:

rust

serde

I have successfully used serde_json to deserialize and serialize JSON. My setup looks somewhat like this (very simplified):

use serde::{Deserialize, Serialize};
use serde_json;
use serde_with::skip_serializing_none;

#[skip_serializing_none]
#[derive(Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
struct Foo {
    #[serde(flatten)]
    bar: Option<Bar>,
    
    baz_quux: Option<u8>,
}

#[skip_serializing_none]
#[derive(Deserialize, Serialize)]
struct Bar {
    #[serde(rename = "plughXyzzySomeRandomStuff")]
    plugh_xyzzy: Option<u8>
}

And then I have implemented FromStr and Display on Foo, which in turn call serde_json::from_str and serde_json::to_string respectively, to easily (de)serialize the struct.

However, I'd now like to also use serde_ini to support (de)serializing INI files as well, to the same Rust data structure. But I can't really figure out how to do that.

The structure itself is simple enough, but my specific problems are with the attributes:

  • The keys are named differently in the JSON and INI formats (the JSON format uses the customary camelCase, while the INI doesn't), so I have to solve the #[serde(rename)] and #[serde(rename_all)] attributes some other way, but I'm not sure where or how.
  • #[serde(flatten)] doesn't seem to work with serde_ini's all-string values, which require a #[serde(deserialize_with="from_str)]" attribute for all non-string values, but this should obviously only apply to the INI values and not the JSON ones.

So all in all, I guess what I need to do is to re-implement these attributes, or use them conditionally based on what (De)Serializer is used, but I'm stumped on how to do that.

like image 473
tobiasvl Avatar asked Oct 23 '21 19:10

tobiasvl


People also ask

What is SerDe serialization?

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically. The Serde ecosystem consists of data structures that know how to serialize and deserialize themselves along with data formats that know how to serialize and deserialize other things.

Is SerDe fast?

Serde. Serde, the incumbent serialization/deserialization library, is elegant, flexible, fast to run, and slow to compile.

Is SerDe slow?

Serde is really designed to be fast, by allowing fully statically dispatched operations without runtime reflections so formats and types are decoupled at code but transparent to the optimization. It's serde_json which cares all the weirdnesses of the JSON format.

What is SerDe JSON?

Serde JSON JSON is a ubiquitous open-standard format that uses human-readable text to transmit data objects consisting of key-value pairs. { "name": "John Doe", "age": 43, "address": { "street": "10 Downing Street", "city": "London" }, "phones": [ "+44 1234567", "+44 2345678" ] }

What types of data can be serialized in serde?

Out of the box, Serde is able to serialize and deserialize common Rust data types in any of the above formats. For example String, &str, usize , Vec<T>, HashMap<K,V> are all supported. In addition, Serde provides a derive macro to generate serialization implementations for structs in your own program.

Why use serde serialization in rust?

This avoids any overhead of reflection or runtime type information. In fact in many situations the interaction between data structure and data format can be completely optimized away by the Rust compiler, leaving Serde serialization to perform the same speed as a handwritten serializer for the specific selection of data structure and data format.

Which data structure knows how to serialize and deserialize itself?

A data structure that knows how to serialize and deserialize itself is one that implements Serde's Serialize and Deserialize traits (or uses Serde's derive attribute to automatically generate implementations at compile time). This avoids any overhead of reflection or runtime type information.

What data formats have been implemented for serde?

The following is a partial list of data formats that have been implemented for Serde by the community. JSON, the ubiquitous JavaScript Object Notation used by many HTTP APIs. Postcard, a no_std and embedded-systems friendly compact binary format.


Video Answer


1 Answers

This is a limitation of serde's design. The Deserialize and Serialize implementations are intentionally separated from the Serializer and Deserializer implementations, which gives great flexibility and convenience when choosing different formats and swapping them out. Unfortunately, it means it is isn't possible to individually fine-tune your Deserialize and Serialize implementations for different formats.

The way I have done this before is to duplicate the data types so that I can configure them for each format, and then provide a zero-cost conversion between them.

like image 77
Peter Hall Avatar answered Oct 26 '22 07:10

Peter Hall