How to deal with completely dynamic JSON responses

Question

Maybe someone in the community has had similar struggles and have come up with a workable solution.

We're currently working on a polyglot key/value store. Given this, we'll generally have no knowledge of what will be stored ahead of time.

Consider the following struct

struct Character : Codable, Equatable {
    let name:    String
    let age:     Int
    let gender:  Gender
    let hobbies: [String]

    static func ==(lhs: Character, rhs: Character) -> Bool {
        return (lhs.name == rhs.name
                   && lhs.age == rhs.age
                   && lhs.gender == rhs.gender
                   && lhs.hobbies == rhs.hobbies)
    }
}

When sending/receiving Character entities over the wire, everything is fairly straight forward. The user can provide us the Type in which we can decode into.

However, we do have the ability to dynamically query the entities stored within the backend. For example, we can request the value of the 'name' property and have that returned.

This dynamism is a pain point. In addition to not knowing the type of the properties outside of the fact that they are Codable, the format that is returned can be dynamic as well.

Here's some examples of response for two different calls extracting properties:

{"value":"Bilbo"}

and

{"value":["[Ljava.lang.Object;",["Bilbo",111]]}

In some cases, it could be an equivalent of a dictionary.

Right now, I have the following structs for dealing with responses:

fileprivate struct ScalarValue<T: Decodable> : Decodable {
    var value: T?
}

Using the Character example, the type passed to the decoder would be:

ScalarValue<Character>.self

However, for the single value, array, or dictionary case, I'm somewhat stuck.

I've started with something like:

fileprivate struct AnyDecodable: Decodable {
    init(from decoder: Decoder) throws {
        // ???
    }
}

Based on the possible return types I've described above, I'm not sure if this is possible with the current API.

Thoughts?

Answer 1

Swift can definitely handle an arbitrary JSON decodable. This isn't the same thing as an arbitrary decodable. JSON can't encode all possible values. But this structure will decode anything that can be expressed in JSON, and from there you can explore it in a type-safe way without resorting to dangerous and awkward tools like Any.

enum JSON: Decodable, CustomStringConvertible {
    var description: String {
        switch self {
        case .string(let string): return "\"\(string)\""
        case .number(let double):
            if let int = Int(exactly: double) {
                return "\(int)"
            } else {
                return "\(double)"
            }
        case .object(let object):
            return "\(object)"
        case .array(let array):
            return "\(array)"
        case .bool(let bool):
            return "\(bool)"
        case .null:
            return "null"
        }
    }

    var isEmpty: Bool {
        switch self {
        case .string(let string): return string.isEmpty
        case .object(let object): return object.isEmpty
        case .array(let array): return array.isEmpty
        case .null: return true
        case .number, .bool: return false
        }
    }

    struct Key: CodingKey, Hashable, CustomStringConvertible {
        var description: String {
            return stringValue
        }

        var hashValue: Int { return stringValue.hash }

        static func ==(lhs: JSON.Key, rhs: JSON.Key) -> Bool {
            return lhs.stringValue == rhs.stringValue
        }

        let stringValue: String
        init(_ string: String) { self.stringValue = string }
        init?(stringValue: String) { self.init(stringValue) }
        var intValue: Int? { return nil }
        init?(intValue: Int) { return nil }
    }

    case string(String)
    case number(Double) // FIXME: Split Int and Double
    case object([Key: JSON])
    case array([JSON])
    case bool(Bool)
    case null

    init(from decoder: Decoder) throws {
        if let string = try? decoder.singleValueContainer().decode(String.self) { self = .string(string) }
        else if let number = try? decoder.singleValueContainer().decode(Double.self) { self = .number(number) }
        else if let object = try? decoder.container(keyedBy: Key.self) {
            var result: [Key: JSON] = [:]
            for key in object.allKeys {
                result[key] = (try? object.decode(JSON.self, forKey: key)) ?? .null
            }
            self = .object(result)
        }
        else if var array = try? decoder.unkeyedContainer() {
            var result: [JSON] = []
            for _ in 0..<(array.count ?? 0) {
                result.append(try array.decode(JSON.self))
            }
            self = .array(result)
        }
        else if let bool = try? decoder.singleValueContainer().decode(Bool.self) { self = .bool(bool) }
        else {
            self = .null
        }
    }

    var objectValue: [String: JSON]? {
        switch self {
        case .object(let object):
            let mapped: [String: JSON] = Dictionary(uniqueKeysWithValues:
                object.map { (key, value) in (key.stringValue, value) })
            return mapped
        default: return nil
        }
    }

    var arrayValue: [JSON]? {
        switch self {
        case .array(let array): return array
        default: return nil
        }
    }

    subscript(key: String) -> JSON? {
        guard let jsonKey = Key(stringValue: key),
            case .object(let object) = self,
            let value = object[jsonKey]
            else { return nil }
        return value
    }

    var stringValue: String? {
        switch self {
        case .string(let string): return string
        default: return nil
        }
    }

    var doubleValue: Double? {
        switch self {
        case .number(let number): return number
        default: return nil
        }
    }

    var intValue: Int? {
        switch self {
        case .number(let number): return Int(number)
        default: return nil
        }
    }

    subscript(index: Int) -> JSON? {
        switch self {
        case .array(let array): return array[index]
        default: return nil
        }
    }

    var boolValue: Bool? {
        switch self {
        case .bool(let bool): return bool
        default: return nil
        }
    }
}

With this, you can do things like:

let bilboJSON = """
{"value":"Bilbo"}
""".data(using: .utf8)!

let bilbo = try! JSONDecoder().decode(JSON.self, from: bilboJSON)
bilbo["value"]  // "Bilbo"

let javaJSON = """
{"value":["[Ljava.lang.Object;",["Bilbo",111]]}
""".data(using: .utf8)!

let java = try! JSONDecoder().decode(JSON.self, from: javaJSON)
java["value"]?[1]   // ["Bilbo", 111]
java["value"]?[1]?[0]?.stringValue  // "Bilbo" (as a String rather than a JSON.string)

The proliferation of ? is somewhat ugly, but using throws on this doesn't really make the interface much nicer in my experiments (particularly because subscripts can't throw). Some tweaking may be advisable based on your particular use cases.