How to use Circe to do a dynamic decoding?

Question

My question is a little tricky. I have a case class looking like this

case class Foo(
    id: String,
    name: String,
    field1: Boolean,
    field2: Boolean,
    field3: Boolean,
    field4: Boolean
)

However, I have two types of input, one is perfectly fit for the case class Foo. The other one is missing values for field3 and field4, looking like {id: "Test", name: "Test", field1: true, field2: true}, I want to create a Decoder[Foo] that works for both cases, if the input is missing field3 and field4, just set a default value false. Is that possible?

For example,

(1) for the input {id: "Test", name: "Test", field1: true, field2: true}, I would like to decode it into

Foo("Test, "Test", true, true, false, flase)

(2) for the input {id: "Test", name: "Test", field1: true, field2: true, field3: true, field4: false}, I would like to decode it into

Foo("Test, "Test", true, true, true, flase)

I know the best solution is to set field3 and field4 as Option[Boolean], however we have a tons of code implemented following the original design and to change the data model will introduce a lot of code change. So just want to see if there is any make shift solution.

thank you very much!

Answer 1

There are multiple ways to do that. I'm going to assume that you are not going to build codecs from scratch and use what you can get from what there is already in circe.

Default parameters + generic-extras

There's circe-generic-extras package, which allows some customization over automatically derived codecs. In particular, it does allow you to use default parameters as fallback values.

The downside is that it's somewhat slower to compile and also requires you to have an implicit io.circe.generic.extras.Configuration in scope.

---

So, first you need that implicit config:

object Configs {
  implicit val useDefaultValues = Configuration.default.withDefaults
}

This usually goes into some generic util package in your project, so you could reuse these configs easily.

Then, you use @ConfiguredJsonCodec macro annotation on your class, or use extras.semiauto.deriveConfiguredCodec in its companion:

import Configs.useDefaultValues

@ConfiguredJsonCodec
case class Foo(
    id: String,
    name: String,
    field1: Boolean,
    field2: Boolean,
    field3: Boolean = false,
    field4: Boolean = false
)

It's important to not forget a config import, and not have more than one config imported at the same time. Otherwise you will get a not helpful error like

could not find Lazy implicit value of type io.circe.generic.extras.codec.ConfiguredAsObjectCodec[Foo]

That's enough to decode Foo now in case fields that have default values are missing:

println {
    io.circe.parser.decode[Foo]("""
{
  "id": "someid",
  "name": "Gordon Freeman",
  "field1": false,
  "field2": true
}
""")
  }

Self-contained scastie here.

Fallback decoder

The idea is as follows: have a separate case class describing the old format of data, and build a decoder to attempt parsing the data as both old and new formats. Circe decoders have or combinator for just that sort of attempting.

---

Here, first you describe the "old" format of data, and a way to upgrade it to a new one:

@JsonCodec(decodeOnly = true)
case class LegacyFoo(
    id: String,
    name: String,
    field1: Boolean,
    field2: Boolean,
) {
  def upgrade: Foo =
    Foo(id, name, field1, field2, false, false)
}

With new format, you have to join the codecs manually, so you can't use macro annotation. Still, you can use generic.semiauto.deriveXXX methods to not have to list all the fields yourself:

case class Foo(
    id: String,
    name: String,
    field1: Boolean,
    field2: Boolean,
    field3: Boolean,
    field4: Boolean
)

object Foo {
  implicit val encoder: Encoder[Foo] = semiauto.deriveEncoder[Foo]
  implicit val decoder: Decoder[Foo] =
    semiauto.deriveDecoder[Foo] or Decoder[LegacyFoo].map(_.upgrade)
}

This will also "just work" for the same payload:

println {
    io.circe.parser.decode[Foo]("""
{
  "id": "someid",
  "name": "Gordon Freeman",
  "field1": false,
  "field2": true
}
""")
}

Scastie here.

---

The first approach requires an extra library, but has less boilerplate. It will also allow the caller to supply, e.g. field4 but not field3 - in second approach, the value of field4 will be entirely discarded in that scenario.

The second one allows to handle more complicated changes than "field added with a default values", like computing values out of several others or changing the structure inside a collection, and also to have several more versions should you need them later.

Oh, you can also put LegacyFoo into object Foo, and make it private if you don't want extra public datatypes exposed.