Remove duplicates from a Json String in Java?

Question

I have a Json String with duplicate values:

String json = "{\"Sign_In_Type\":\"Action\",\"Sign_In_Type\":\"Action\"}";

that correctly throws an exception when I try to create a JSONObject:

   try {
            JSONObject json_obj = new JSONObject(json);
            String type = json_obj.getString("Sign_In_Type");
        } catch (JSONException e) {
            throw new RuntimeException(e);
        }

Error:

Exception in thread "main" java.lang.RuntimeException: org.json.JSONException: Duplicate key "Sign_In_Type"
    at com.campanja.app.Upload.main(Upload.java:52)
Caused by: org.json.JSONException: Duplicate key "Sign_In_Type"
    at org.json.JSONObject.putOnce(JSONObject.java:1076)
    at org.json.JSONObject.(JSONObject.java:205)
    at org.json.JSONObject.(JSONObject.java:402)
    at com.campanja.app.Upload.main(Upload.java:49)

Is there a smart way of removing or checking for duplicates before I convert it to a JSONOBject? I have tried to create:

 Set set = new HashSet(Arrays.asList(json));

but that gives me:

[{"Sign_In_Type":"Action","Sign_In_Type":"Action"}]

Any suggesstions welcome, thanks!

Answer 1

Two options I can think of right off the bat:

• Parse the string using wither regex or tokens, add each key-value pair to a hashmap, and in the end recreate your JSON document with the duplicates removed. In this case though I would only remove key-value pairs that are exactly the same.
• Download the source code for org.json.JSONObject , and make a slight modification to the code to automatically leave out duplicates. This is a bit dangerous though. Another option is to create a modified version that simply validates and modifies.

Extending JSONObject Working Example

The below code allows you to create a JSONOBbject with a string containing duplicate keys. Exceptions are thrown only when you have two key-values that have the same key, but different values. This was because I think it would be a problem to choose at random which of the two should be assigned (e.g. the later value?). Of course this can be changed to work as you wish (e.g. keep last value for multiple keys).

Modified Class

import org.json.JSONException;
import org.json.JSONObject;


public class JSONObjectIgnoreDuplicates extends JSONObject {

     public JSONObjectIgnoreDuplicates(String json) {
        super(json);
    }

    public JSONObject putOnce(String key, Object value) throws JSONException {
            Object storedValue;
            if (key != null && value != null) {
                if ((storedValue = this.opt(key)) != null ) {
                    if(!storedValue.equals(value))                          //Only through Exception for different values with same key
                        throw new JSONException("Duplicate key \"" + key + "\"");
                    else
                        return this;
                }
                this.put(key, value);
            }
            return this;
        }
}

Main method

String json = "{\"Sign_In_Type\":\"Action\",\"Sign_In_Type\":\"Action\"}";
           try {
                JSONObject json_obj = new JSONObjectIgnoreDuplicates(json);
                String type = json_obj.getString("Sign_In_Type");
            } catch (JSONException e) {
                throw new RuntimeException(e);
            }