Retrofit - removing some invalid characters from response body before parsing it as json

Question

I have an external web service that in the response body returns json but nested in parentheses, like this:

({"door_x":"103994.001461","door_y":"98780.7862376", "distance":"53.3"})

Using this code:

class AddressInfo {
    String door_x;
    String door_y;
}

interface AddressWebService {
    @GET("/reversegeocoding")
    AddressInfo reverseGeocoding(@Query("x") double x, @Query("y") double y);
}

It obviously fails. This is the stacktrace:

retrofit.RetrofitError: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
        at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:377)
        at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
        at com.something.$Proxy7.reverseGeocoding(Native Method)
        at com.something.ReverseGeocodingService.getAddress(ReverseGeocodingService.java:24)
        at com.something.LocationProvider$1.run(LocationProvider.java:77)
        at java.lang.Thread.run(Thread.java:864)
 Caused by: retrofit.converter.ConversionException: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
        at retrofit.converter.GsonConverter.fromBody(GsonConverter.java:67)
        at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:362)
            at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
            at com.something.$Proxy7.reverseGeocoding(Native Method)
            at com.something.ReverseGeocodingService.getAddress(ReverseGeocodingService.java:24)
            at com.something.LocationProvider$1.run(LocationProvider.java:77)
            at java.lang.Thread.run(Thread.java:864)
 Caused by: com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
        at com.google.gson.Gson.fromJson(Gson.java:803)
        at com.google.gson.Gson.fromJson(Gson.java:768)
        at retrofit.converter.GsonConverter.fromBody(GsonConverter.java:63)
            at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:362)
            at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
            at com.something.$Proxy7.reverseGeocoding(Native Method)
            at com.something.ReverseGeocodingService.getAddress(ReverseGeocodingService.java:24)
            at com.something.LocationProvider$1.run(LocationProvider.java:77)
            at java.lang.Thread.run(Thread.java:864)
 Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1
        at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:374)
        at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:165)
            at com.google.gson.Gson.fromJson(Gson.java:803)
            at com.google.gson.Gson.fromJson(Gson.java:768)
            at retrofit.converter.GsonConverter.fromBody(GsonConverter.java:63)
            at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:362)
            at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
            at com.something.$Proxy7.reverseGeocoding(Native Method)
            at com.something.ReverseGeocodingService.getAddress(ReverseGeocodingService.java:24)
            at com.something.LocationProvider$1.run(LocationProvider.java:77)
            at java.lang.Thread.run(Thread.java:864)

What is the best way to remove the parentheses before parsing the json?

Answer 1

You can clean painlessly the response in your GsonConverter before Gson deserialized the body into type object.

 public class CleanGsonConverter extends GsonConverter{

            public CleanGsonConverter(Gson gson) {
                super(gson);
            }

            public CleanGsonConverter(Gson gson, String encoding) {
                super(gson, encoding);
            }

            @Override
            public Object fromBody(TypedInput body, Type type) throws ConversionException {
                String dirty = toString(body);
                String clean = dirty.replaceAll("(^\\(|\\)$)", "");
                body = new JsonTypedInput(clean.getBytes(Charset.forName(HTTP.UTF_8)));
                return super.fromBody(body, type);
            }
            private String toString(TypedInput body){
                    BufferedReader br = null;
                    StringBuilder sb = new StringBuilder();
                    String line;
                    try {
                        br = new BufferedReader(new InputStreamReader(body.in()));
                        while ((line = br.readLine()) != null) {
                            sb.append(line);
                        }

                    } catch (IOException e) {
                        e.printStackTrace();
                    } finally {
                        if (br != null) {
                            try {
                                br.close();
                            } catch (IOException e) {
                                e.printStackTrace();
                            }
                        }
                    }

                    return sb.toString();

                }
        };

JsonTypedInput:

   public class JsonTypedInput implements TypedInput{

        private final byte[] mStringBytes;

        JsonTypedInput(byte[] stringBytes) {
            this.mStringBytes = stringBytes;
        }


        @Override
        public String mimeType() {
            return "application/json; charset=UTF-8";
        }



        @Override
        public long length() {
            return mStringBytes.length;
        }

        @Override
        public InputStream in() throws IOException {
            return new ByteArrayInputStream(mStringBytes);
        }
    }

Here I subclassed GsonConverter to get access to the response before it is converted to object. JsonTypedOutput is used to preserve the mime type of the response after cleaning it from the junk chars.

Usage:

restAdapterBuilder.setConverter(new CleanGsonConverter(gson));

Blame it on your backend guys. :)

Answer 2

Solution For Retrofit 2

The code below is the same as the GsonConverter except you can edit the Response before converting to its model

Edit public T convert(ResponseBody value) to clean your Response

/**
 * Modified by TarekkMA on 8/2/2016.
 */

public class MyJsonConverter extends Converter.Factory {

    public static MyJsonConverter create() {
        return create(new Gson());
    }

    public static JsonHandler create(Gson gson) {
        return new JsonHandler(gson);
    }

    private final Gson gson;

    private JsonHandler(Gson gson) {
        if (gson == null) throw new NullPointerException("gson == null");
        this.gson = gson;
    }

    @Override
    public Converter<ResponseBody, ?> responseBodyConverter(Type type, Annotation[] annotations,
                                                            Retrofit retrofit) {
        TypeAdapter<?> adapter = gson.getAdapter(TypeToken.get(type));
        return new GsonResponseBodyConverter<>(gson, adapter);
    }

    @Override
    public Converter<?, RequestBody> requestBodyConverter(Type type,
                                                          Annotation[] parameterAnnotations, Annotation[] methodAnnotations, Retrofit retrofit) {
        TypeAdapter<?> adapter = gson.getAdapter(TypeToken.get(type));
        return new GsonRequestBodyConverter<>(gson, adapter);
    }


    final class GsonRequestBodyConverter<T> implements Converter<T, RequestBody> {
        private final MediaType MEDIA_TYPE = MediaType.parse("application/json; charset=UTF-8");
        private final Charset UTF_8 = Charset.forName("UTF-8");

        private final Gson gson;
        private final TypeAdapter<T> adapter;

        GsonRequestBodyConverter(Gson gson, TypeAdapter<T> adapter) {
            this.gson = gson;
            this.adapter = adapter;
        }

        @Override
        public RequestBody convert(T value) throws IOException {
            Buffer buffer = new Buffer();
            Writer writer = new OutputStreamWriter(buffer.outputStream(), UTF_8);
            JsonWriter jsonWriter = gson.newJsonWriter(writer);
            adapter.write(jsonWriter, value);
            jsonWriter.close();
            return RequestBody.create(MEDIA_TYPE, buffer.readByteString());
        }
    }

    final class GsonResponseBodyConverter<T> implements Converter<ResponseBody, T> {
        private final Gson gson;
        private final TypeAdapter<T> adapter;

        GsonResponseBodyConverter(Gson gson, TypeAdapter<T> adapter) {
            this.gson = gson;
            this.adapter = adapter;
        }

        @Override
        public T convert(ResponseBody value) throws IOException {
            String dirty = value.string();
            String clean = dirty.replace("<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n" +
                    "<string xmlns=\"http://tempuri.org/\">","").replace("</string>","");
            try {
                return adapter.fromJson(clean);
            } finally {
                value.close();
            }
        }
    }


}

• Another solution is to blame the inexperienced backend developer.