Decoding a Java/JSON Map into an F# object

Question

I'm having trouble converting a Java/JSON map into a usable F# object.

Here's the heart of my code:

member this.getMapFromRpcAsynchronously =
    Rpc.getJavaJSONMap (new Action<_>(this.fillObjectWithJSONMap))
    ()

member this.fillObjectWithJSONMap (returnedMap : JSONMap<string, int> option) = 
    let container = Option.get(returnedMap)
    let map = container.map
    for thing in map do
        this.myObject.add thing.key
        // do stuff with thing
    ()

The JSON that's returned by my RPC method looks like this:

{"id":1, "result":
    {"map":
        {"Momentum":12, "Corporate":3, "Catalyst":1},
     "javaClass":"java.util.HashMap"}
}

I'm attempting to map it to an F# DataContract which looks like this:

[<DataContract>]
type JSONMap<'T, 'S> = {
    [<DataMember>]
    mutable map : KeyValuePair<'T, 'S> array
    [<DataMember>]
    mutable javaClass : string
}

[<DataContract>]
type JSONSingleResult<'T> = {
    [<DataMember>]
    mutable javaClass: string
    [<DataMember>]
    mutable result: 'T
}

Finally, the F# method that performs the actual RPC call (Rpc.getJavaJSONMap above) looks like this:

let getJavaJSONMap (callbackUI : Action<_>) = 
    ClientRpc.getSingleRPCResult<JSONSingleResult<JSONMap<string, int>>, JSONMap<string, int>>
        "MyJavaRpcClass"
        "myJavaRpcMethod"
        "" // takes no parameters
        callbackUI
        (fun (x : option<JSONSingleResult<JSONMap<string, int>>>) ->
            match x.IsSome with
                | true -> Some(Option.get(x).result)
                | false -> None 
        )

At compile time I get no errors. My RPC method is called, and a result is returned (using Fiddler to see the actual call & return). However, it appears the F# is having trouble matching the JSON into my DataContract, since returnedMap at the very top is always null.

Any thoughts or advice would be greatly appreciated. Thank you.

Answer 1

Here's what I cooked up:

open System.Web.Script.Serialization   // from System.Web.Extensions assembly

let s = @"
    {""id"":1, ""result"": 
        {""map"": 
            {""Momentum"":12, ""Corporate"":3, ""Catalyst"":1}, 
         ""javaClass"":""java.util.HashMap""} 
    } 
    "

let jss = new JavaScriptSerializer()
let o = jss.DeserializeObject(s)

// DeserializeObject returns nested Dictionary<string,obj> objects, typed
// as 'obj'... so add a helper dynamic-question-mark operator
open System.Collections.Generic 
let (?) (o:obj) name : 'a = (o :?> Dictionary<string,obj>).[name] :?> 'a

printfn "id: %d" o?id
printfn "map: %A" (o?result?map 
                   |> Seq.map (fun (KeyValue(k:string,v)) -> k,v) 
                   |> Seq.toList)
// prints:
// id: 1
// map: [("Momentum", 12); ("Corporate", 3); ("Catalyst", 1)]

Answer 2

Hmm this is a complicated problem. I assume this:

{"map": 
        {"Momentum":12, "Corporate":3, "Catalyst":1}, 
     "javaClass":"java.util.HashMap"} 

could contain a variable amount of fields. And in JSON notation translates to an object (javascript objects are basically (or very similar) to maps). I don't know if this will translate to F# directly.

It might be prevented not be allowed by F# static typing versus the javascript's dynamic typing.

you may have to write the conversion routine yourself.

---

Ok there are a couple of small bugs in the the data contracts lets redefine the JsonMap and remove the "javaclass" attribute as it is not in th JSON sample provided (it is a higher level up), and it looks as though the keyvaulepair to me is not serializing, so lets define our own type:

type JsonKeyValuePair<'T, 'S> =  {
    [<DataMember>] 
    mutable key : 'T
    [<DataMember>] 
    mutable value : 'S
}

type JSONMap<'T, 'S> = { 
    [<DataMember>] 
    mutable map : JsonKeyValuePair<'T, 'S> array 
} 

and create a deserialize function:

let internal deserializeString<'T> (json: string)  : 'T = 
    let deserializer (stream : MemoryStream) = 
        let jsonSerializer 
            = Json.DataContractJsonSerializer(
                typeof<'T>)
        let result = jsonSerializer.ReadObject(stream)
        result


    let convertStringToMemoryStream (dec : string) : MemoryStream =
        let data = Encoding.Unicode.GetBytes(dec); 
        let stream = new MemoryStream() 
        stream.Write(data, 0, data.Length); 
        stream.Position <- 0L
        stream

    let responseObj = 
        json
            |> convertStringToMemoryStream
            |> deserializer

    responseObj :?> 'T


let run2 () = 
    let json = "{\"map@\":[{\"key@\":\"a\",\"value@\":1},{\"key@\":\"b\",\"value@\":2}]}"
    let o  = deserializeString<JSONMap<string, int>> json
    ()

I am able to deserialize a string into the appropriate object structure. Two things I would like to see answered are

1) why is .NET forcing me to append @ characters after the field names? 2) what is the best way to do the conversion? I would guess an abstract syntax tree representing the JSON structure might be the way to go, and then to parse that into the new string. I am not super familiar with AST and their parsing though.

Perhaps one of the F# experts might be able to help or come up with a better translation scheme?

---

lastly adding back in the result type:

[<DataContract>] 
type Result<'T> = { 
    [<DataMember>] 
    mutable javaClass: string 
    [<DataMember>] 
    mutable result: 'T 
} 

and a convert map function (works in this case - but has many areas of weakness including recursive map definitions etc):

let convertMap (json: string) = 
    let mapToken = "\"map\":"
    let mapTokenStart = json.IndexOf(mapToken)
    let mapTokenStart  = json.IndexOf("{", mapTokenStart)
    let mapObjectEnd  = json.IndexOf("}", mapTokenStart)
    let mapObjectStart = mapTokenStart
    let mapJsonOuter = json.Substring(mapObjectStart, mapObjectEnd - mapObjectStart + 1)
    let mapJsonInner = json.Substring(mapObjectStart + 1, mapObjectEnd - mapObjectStart - 1)
    let pieces = mapJsonInner.Split(',')
    let convertPiece state (piece: string) = 
        let keyValue = piece.Split(':')
        let key = keyValue.[0]
        let value = keyValue.[1]
        let newPiece = "{\"key\":" + key + ",\"value\":" + value + "}"
        newPiece :: state

    let newPieces = Array.fold convertPiece []  pieces
    let newPiecesArr = List.toArray newPieces
    let newMap = String.Join(",",  newPiecesArr)
    let json = json.Replace(mapJsonOuter, "[" + newMap + "]")
    json



let json = "{\"id\":1, \"result\": {\"map\": {\"Momentum\":12, \"Corporate\":3, \"Catalyst\":1}, \"javaClass\":\"java.util.HashMap\"} } "
printfn <| Printf.TextWriterFormat<unit>(json)
let json2 = convertMap json
printfn <| Printf.TextWriterFormat<unit>(json2)
let obj = deserializeString<Result<JSONMap<string,int>>> json2

It still inisits on the @ sign in various places - which I don't get...

---

adding convert w/ workaround for the ampersand issue

let convertMapWithAmpersandWorkAround (json: string) = 
    let mapToken = "\"map\":"
    let mapTokenStart = json.IndexOf(mapToken)
    let mapObjectEnd  = json.IndexOf("}", mapTokenStart)
    let mapObjectStart = json.IndexOf("{", mapTokenStart)
    let mapJsonOuter = json.Substring(mapTokenStart , mapObjectEnd - mapTokenStart + 1)
    let mapJsonInner = json.Substring(mapObjectStart + 1, mapObjectEnd - mapObjectStart - 1)
    let pieces = mapJsonInner.Split(',')
    let convertPiece state (piece: string) = 
        let keyValue = piece.Split(':')
        let key = keyValue.[0]
        let value = keyValue.[1]
        let newPiece = "{\"key@\":" + key + ",\"value@\":" + value + "}"
        newPiece :: state

    let newPieces = Array.fold convertPiece []  pieces
    let newPiecesArr = List.toArray newPieces
    let newMap = String.Join(",",  newPiecesArr)
    let json = json.Replace(mapJsonOuter, "\"map@\":[" + newMap + "]")
    json



let json = "{\"id\":1, \"result\": {\"map\": {\"Momentum\":12, \"Corporate\":3, \"Catalyst\":1}, \"javaClass\":\"java.util.HashMap\"} } "
printfn <| Printf.TextWriterFormat<unit>(json)
let json2 = convertMapWithAmpersandWorkAround json
printfn <| Printf.TextWriterFormat<unit>(json2)
let obj = deserialize<Result<JSONMap<string,int>>> json2

---

adding:

[<DataContract>] 

above the record fixes the Ampersand issue.