Use circe to preprocess dot-notation style fields

Question

I have some json that includes some fields that are being flattened into a bson-ish format as in {"foo.bar" : "bash"}. I'd like to transform this to the following representation {"foo" : { "bar" : "bash"}} and wondering where in circe I'd do such an operation. Complicating the problem is that there could be multiple such fields that need to be properly merged, e.g. {"foo.bar" : "a", "foo.bash" : "b", "foo.baz" : "c"} -> {"foo" : { "bar" : "a", "bash" : "b", "baz" : "c"}}.

Answer 1

Here's a quick implementation:

import io.circe.Json

val Dotted = "([^\\.]*)\\.(.*)".r

def expandDotted(j: Json): Json = j.arrayOrObject(
  j,
  js => Json.fromValues(js.map(expandDotted)),
  _.toList.map {
    case (Dotted(k, rest), v) => Json.obj(k -> expandDotted(Json.obj(rest -> v)))
    case (k, v) => Json.obj(k -> expandDotted(v))
  }.reduceOption(_.deepMerge(_)).getOrElse(Json.obj())
)

I haven't really used or tested it in detail, but it seems to work:

scala> import io.circe.literal._
import io.circe.literal._

scala> val j1 = json"""{"foo.bar" : "a", "foo.bash" : "b", "foo.baz" : "c"}"""
j1: io.circe.Json =
{
  "foo.bar" : "a",
  "foo.bash" : "b",
  "foo.baz" : "c"
}

scala> expandDotted(j1)
res1: io.circe.Json =
{
  "foo" : {
    "baz" : "c",
    "bash" : "b",
    "bar" : "a"
  }
}

And with deeper nesting:

scala> expandDotted(json"""{ "x.y.z": true, "a.b": { "c": 1 } }""")
res2: io.circe.Json =
{
  "a" : {
    "b" : {
      "c" : 1
    }
  },
  "x" : {
    "y" : {
      "z" : true
    }
  }
}

And just to confirm that it doesn't mess with undotted keys:

scala> expandDotted(json"""{ "a.b": true, "x": 1 }""").noSpaces
res3: String = {"x":1,"a":{"b":true}}

Note that in the case of "collisions" (paths that lead to both JSON objects and non-object JSON values, or to multiple non-object values), the behavior is that of Json#deepMerge:

scala> expandDotted(json"""{ "a.b": true, "a": 1 }""").noSpaces
res4: String = {"a":1}

scala> expandDotted(json"""{ "a": 1, "a.b": true }""").noSpaces
res5: String = {"a":{"b":true}}

…which is probably what you'd want, but you could also have it fail in these cases, or not expand the colliding path, or do pretty much any other thing you can think of.