Flatten an arbitrarily nested codec?

Question

As a new user of SCodec, there is quite a learning curve. I've hit a snag that I can't seem to solve despite reading the source and docs.

I want to be able to define popular codecs as functions like this

def packedByte : Codec[Int :: Int :: Int :: HNil] = uint(4) :: uint(2) :: uint(2)

And then combine them in to higher level codecs like this which decode to and encode from case classes like this

case class MyPacket(foo : Boolean, first : Int, second : Int, third : Int, bar : Boolean)
def packet : Codec[MyPacket] = (bool :: packedByte :: bool).as[MyPacket]

But, this doesn't work saying

Could not prove that shapeless.::[Boolean,shapeless.::[shapeless.::[Int,shapeless.::[Int,shapeless.::[Int,shapeless.HNil]]],shapeless.::[Boolean,shapeless.HNil]]] can be converted to/from cmd504.MyPacket.

Yet, when I "inline" the packedByte, like

def packetInline : Codec[MyPacket] = (bool :: uint(4) :: uint(2) :: uint(2) :: bool).as[MyPacket]

Everything compiles and works as expected. My intuition tells me that the Codec must be "flattened" (based off of the two HNils in the error message), but I have been unable to flatten the Codec itself or the internal HList representation.

Answer 1

It's often useful to start reasoning about hlists by thinking about how you'd work with ordinary value-level lists in a similar situation. For example, suppose we've got a value and a list:

val x = 0
val xs = List(1, 2, 3)

And we want to create a new list with x both before and after xs. We can use +: and :+:

scala> x +: xs :+ x
res0: List[Int] = List(0, 1, 2, 3, 0)

Or:

scala> x :: (xs :+ x)
res1: List[Int] = List(0, 1, 2, 3, 0)

In the case of Scodec, there's no +: operator, but there are :: and :+, and you can use them exactly as you would use the list versions at the value level:

import scodec._, scodec.codecs._, shapeless._

def packedByte: Codec[Int :: Int :: Int :: HNil] =
  uint(4) :: uint(2) :: uint(2)

case class MyPacket(
  foo: Boolean,
  first: Int,
  second: Int,
  third: Int,
  bar: Boolean
)

def packet: Codec[MyPacket] = (bool :: (packedByte :+ bool)).as[MyPacket]

It would be possible to construct a nested hlist and then flatten it, but :+ is far more idiomatic.