Elm decoding unknown json structure

Question

I've just started working with Elm to do some front-end prototyping using a Rest API I'm working on. In general, the API returns "reasonable" data structures that can be decoded because the keys and value-types are well-known, but several resource types return a data entry that just has raw json that has no predetermined structure.

Everything I've read so far seems to assume you know the structure of the data you're decoding, whereas in plain js it's relatively easy to loop over the keys and reflect on the types in order to determine how they should be handled at runtime. I'm not seeing a clear path toward handling this kind of data in Elm yet.

E.g.,

{
  "name":"foo",
  "data": {
    "bar": [{"baz":123}, "quux"]
  },
  ...
}

I'd like to know if it is currently possible to parse the value of the data entry with something akin to

function go(obj)
    for key in keys(foo)
        if foo[key] is an object
            go(foo[k])
        else if foo[key] is an array
            map(go, foo[k])
        ...

Specifically:

1. Is it currently possible to handle unknown, possibly deeply nested and heterogeneous json data in Elm?
2. If so, can you give me the key concept or high level intuition on how the author(s) intended data like this to be decoded?

Answer 1

Yes, it is possible to write a general purpose decoder. You can first define a union type that holds all possible Json types:

type JsVal
  = JsString String
  | JsInt Int
  | JsFloat Float
  | JsArray (List JsVal)
  | JsObject (Dict String JsVal)
  | JsNull

And now you can use Json.Decode.oneOf to try every possibility.

import Json.Decode as D exposing (Decoder)
import Dict exposing (Dict)

jsValDecoder : Decoder JsVal
jsValDecoder =
  D.oneOf
    [ D.string |> D.andThen (D.succeed << JsString)
    , D.int |> D.andThen (D.succeed << JsInt)
    , D.float |> D.andThen (D.succeed << JsFloat)
    , D.list (D.lazy (\_ -> jsValDecoder)) |> D.andThen (D.succeed << JsArray)
    , D.dict (D.lazy (\_ -> jsValDecoder)) |> D.andThen (D.succeed << JsObject)
    , D.null JsNull
    ]

Json.Decode.lazy is necessary for the JsArray and JsObject constructors because they are defined recursively.

This structure should handle anything you throw at it, and it will be up to the rest of your program to decide what to do with such a flexible type.

Edit

As @Tosh pointed out, this decoder can be cleaned up by using map instead of an andThen followed by a succeed:

jsValDecoder : Decoder JsVal
jsValDecoder =
  D.oneOf
    [ D.map JsString D.string
    , D.map JsInt D.int
    , D.map JsFloat D.float
    , D.list (D.lazy (\_ -> jsValDecoder)) |> D.map JsArray
    , D.dict (D.lazy (\_ -> jsValDecoder)) |> D.map JsObject
    , D.null JsNull
    ]

Answer 2

In Chad's excellent answer, the boolean type is missing. Here's a full module able to handle booleans as well:

module Data.JsonValue exposing (JsonValue(..), decoder)

import Dict exposing (Dict)
import Json.Decode as Decode
    exposing
        ( Decoder
        , dict
        , string
        , int
        , float
        , list
        , null
        , oneOf
        , lazy
        , map
        , bool
        )


type JsonValue
    = JsonString String
    | JsonInt Int
    | JsonFloat Float
    | JsonBoolean Bool
    | JsonArray (List JsonValue)
    | JsonObject (Dict String JsonValue)
    | JsonNull


decoder : Decoder JsonValue
decoder =
    oneOf
        [ map JsonString string
        , map JsonInt int
        , map JsonFloat float
        , map JsonBoolean bool
        , list (lazy (\_ -> decoder)) |> map JsonArray
        , dict (lazy (\_ -> decoder)) |> map JsonObject
        , null JsonNull
        ]