URL class rendering - Invalid_Request with next_page_token

Question

In my Android application the user can select categories (cafe, restaurant, ...) of which places he wants to see.

I get result from the google-places-api with a next_page_token, when I use the following url:

https://maps.googleapis.com/maps/api/place/textsearch/json?query=cafe+in+Elfde-Liniestraat%2C+Hasselt%2C+Belgium&pagetoken=ClRHAAAAqYa7IpwMQH3q1DyinfsMnkuOaZg8wQrp0pvzarv13pwqruqT2YfaEry6v_d-L4d_TC13G6iHuPz-rc1KbI6XRE01rB5CuXcHISu-CI_CaKESEMUxSjHhHDNee4xbr2tPEZcaFPvO6xlRDjs-5AixyD9vamWwTGlU&sensor=true&key=<mykey>

or this url:

https://maps.googleapis.com/maps/api/place/textsearch/json?pagetoken=ClRHAAAAqYa7IpwMQH3q1DyinfsMnkuOaZg8wQrp0pvzarv13pwqruqT2YfaEry6v_d-L4d_TC13G6iHuPz-rc1KbI6XRE01rB5CuXcHISu-CI_CaKESEMUxSjHhHDNee4xbr2tPEZcaFPvO6xlRDjs-5AixyD9vamWwTGlU&sensor=true&key=<mykey>

I get a INVALID_REQUEST.

This is the code I use to run and fetch the results:

@Override
    protected ArrayList<String> doInBackground(String... params) {
        // When more then one category is selected, this character needs to be after it.
        final String DELIMITINGCHARACTER = "|";

        // Add all the categories to a variable to put later on in the URL.
        String myTypes = "";
        for (String category : params){
            // A category must be lowercase letters only.
            category = category.toLowerCase();
            // A category may not display a whitespace.
            // The whitespace(s) must be replaced with a '_'.
            category = category.replaceAll(" ", "_");

            myTypes += category + DELIMITINGCHARACTER;
        }   

        // Delete if the string ends with the delimitingcharachter the delimitingcharacter!
        if (myTypes.endsWith("|")){
            myTypes = myTypes.substring(0, myTypes.length() - 1);
        }

        ArrayList<String> myPointsOfInterest = new ArrayList<String>();
        try{ 
            String urlString;
            String urlData;
            //String urlApi = "&sensor=true&key=<myKey>"; // Key 1
            String urlApi = "&sensor=true&key=<myKey>"; // Key 2

            if (_txtFindLocation.getText().length() > 0){
                urlString = "maps.googleapis.com/maps/api/place/textsearch/json?";
                urlData = "query=" + URLEncoder.encode(myTypes, "UTF-8") + "+in+" 
                        + URLEncoder.encode(_txtFindLocation.getText().toString(), "UTF-8");
            }
            else{
                urlString = "maps.googleapis.com/maps/api/place/search/json?";
                urlData = "location=" + _myCurrentPosition.getLatitudeE6() / 1e6 + "," + _myCurrentPosition.getLongitudeE6() / 1e6
                        + "&radius=5000&types=" + URLEncoder.encode(myTypes, "UTF-8");
            }

            ArrayList<String> myJsonString = new ArrayList<String>();

            myJsonString.add(getDataOfUrl(urlString, urlData, urlApi, null));

            for (String jsonString : myJsonString){
                // Turn String into a JSON Object.
                JSONObject jsonPredictionsObject = new JSONObject(jsonString);

                String urlToken = jsonPredictionsObject.optString("next_page_token");

                if (urlToken != ""){
                    myJsonString.add(getDataOfUrl(urlString, urlData, urlApi, "&pagetoken=" + urlToken));
                }
            }
            Log.d("MainActivity", "size of myJsonString " + String.valueOf(myJsonString.size()));
            for (String jsonString : myJsonString){
                Log.d("MainActivity", jsonString);
                // Turn String into a JSON Object.
                JSONObject jsonPredictionsObject = new JSONObject(jsonString);

                // Get a JSON Array that is inside the JSON Object.
                JSONArray jsonPredictionsArray = jsonPredictionsObject.getJSONArray("results");

                for (int i = 0; i < jsonPredictionsArray.length(); i++){
                    jsonPredictionsObject = (JSONObject) jsonPredictionsArray.get(i);

                    final String name = jsonPredictionsObject.getString("name");
                    final String vicinity = jsonPredictionsObject.optString("vicinity").compareTo("") != 0 ? jsonPredictionsObject.optString("vicinity") : jsonPredictionsObject.optString("formatted_address");
                    final JSONObject geometryObject = jsonPredictionsObject.getJSONObject("geometry");
                    final JSONObject locationObject = geometryObject.getJSONObject("location");
                    final JSONArray typesArray = jsonPredictionsObject.getJSONArray("types");
                    final double latitude = locationObject.getDouble("lat");
                    final double longitude = locationObject.getDouble("lng");

                    String types = "";
                    for (int t = 0; t < typesArray.length(); t++){
                        String type = typesArray.getString(t);

                        type = type.toLowerCase();
                        type = type.replaceAll("_", " ");

                        types += type + " - ";
                    }

                    types = types.substring(0, types.length() - 2);

                    myPointsOfInterest.add(String.valueOf(latitude) + DELIMITINGCHARACTER + 
                            String.valueOf(longitude) + DELIMITINGCHARACTER + types + 
                            DELIMITINGCHARACTER + name + DELIMITINGCHARACTER + vicinity);
                }
            }
        }
        catch (IOException e){
            Log.e("Search points of interest", "GetPointsOfinterest: doInBackGround", e);
        }
        catch (JSONException e){
            Log.e("Search points of interest", "GetPointsOfinterest: doInBackGround", e);
        }

        return myPointsOfInterest;
    }


public String getDataOfUrl(String urlString, String urlData, String urlApi, String urlToken){
    try{
        URL myURL;

        if (urlToken == null) {
            myURL = new URL("https://" + urlString + urlData + urlApi);
        }
        else {
            Log.d("URL INPUT", "https://" + urlString + urlData + urlToken + urlApi);
            myURL = new URL("https://" + urlString + urlData + urlToken + urlApi);
        }

        URLConnection myURLConnection = myURL.openConnection();         
        BufferedReader myBufferReader = new BufferedReader(new InputStreamReader(myURLConnection.getInputStream()));

        String strLine;
        StringBuffer strBuffer = new StringBuffer();
        // Take Google's legible JSON and turn it into on big String.
        while((strLine = myBufferReader.readLine()) != null){
            strBuffer.append(strLine);
        }           
        Log.d("URL OUTPUT", strBuffer.toString());
        return strBuffer.toString();
    }
    catch (IOException e){
        Log.e("getDataOfUrl", "Error: ", e);
    }

    return null;
}

This is the output of logcat:

11-03 14:47:13.832: D/URL OUTPUT(19532): {   "html_attributions" : [],   "next_page_token" : "ClRHAAAAqYa7IpwMQH3q1DyinfsMnkuOaZg8wQrp0pvzarv13pwqruqT2YfaEry6v_d-L4d_TC13G6iHuPz-rc1KbI6XRE01rB5CuXcHISu-CI_CaKESEMUxSjHhHDNee4xbr2tPEZcaFPvO6xlRDjs-5AixyD9vamWwTGlU",   "results" : [      {         "formatted_address" : "Meldertstraat 42, Hasselt, België",         "geometry" : {            "location" : {               "lat" : 50.9311660,               "lng" : 5.3423240            }         },         "icon" : "http://maps.gstatic.com/mapfiles/place_api/icons/cafe-71.png",         "id" : "e6ba73e5b439dddb83aaefc94c71954da0cb0268",         "name" : "Café Café",         "rating" : 4.40,         "reference" : "CnRpAAAAdVZpUI4NKwsAiQTOjbYG9psNbU7awZkS6_8Ee4r3AvCKhpqWeiDqdof7zYmoUZWKj0A8rOwtncSa1YFVNNo6RxFmmHvuNdWCaucTqkuSSDw8E2o0pf5E6EiMj-7-Hd5xBzDtC0j9d5lLcVWW3AQy1xIQRwgfWwrZkyK65blCeGqhFhoUB9kTW5QVD9sNNuF75MvVewYTcxA",         "types" : [ "cafe", "food", "establishment" ]      },      {         "formatted_address" : "Kempischesteenweg 27, Hasselt, België",         "geometry" : {            "location" : {               "lat" : 50.9350770,               "lng" : 5.3397760            }         },         "icon" : "http://maps.gstatic.com/mapfiles/place_api/icons/cafe-71.png",         "id" : "dc66d311b3c4f5d1a87d5959dfdbc7616ad10b24",         "name" : "Oberonn",         "opening_hours" : {            "open_now" : true         },         "reference" : "CnRlAAAA5q-rz_aOTWsFGQIJz_6zyLbXIczXOTq4PVFxniOjxsCwnw7uIfY8tu7Jk-QbJ-arMWgdClnSpX28q2SqxD8021LGWcaf4Sgo9MaXWpoD6c4TSagyVO9_l_HUXgMoOFJzhHG_NH6z-t1DPxT1zD19PhIQ2gBP9cLazKeB96ZDTpFdsxoUVGB2Ddhrq6JUoTJgrHGL-SINOWk",         "types" : [ "cafe", "food", "establishment" ]      },      {         "formatted_address" : "Willekensmolenstraat 41, Hasselt, België",         "geometry" : {            "location" : {               "lat" : 50.9316830,               "lng" : 5.3470750            }         },         "icon" : "http://maps.gstatic.com/mapfiles/place_api/icons/cafe-71.png",         "id" : "04937541c2bf71ebec7ac13d506dbbd2727e1ce3",         "name" : "Den Drossaerd",         "reference" : "CnRqAAAA0cPmuMx0_J4x-o-yn0oT1D3kD-IHYAQPotAPPeDD3DWh2FXkD89kCsYoUGDLBlH2mKY6ju18JyCHvc4W_izLcd6TUOIoXrbUCyrtxOL4sTAFb41kH1aFT4dOBgvigYrGIcV8Qbzq-K4YGL3t7pnf1hIQKmUp-LLqtGHb4waYi2nq8hoU3VgwykpytoSUUPe2Ue1HmPrMRtE",         "types" : [ "cafe", "food", "establishment" ]      },      {         "formatted_address" : "Leopoldplein 1, Hasselt, België",         "geometry" : {            "location" : {               "lat" : 50.9275810,               "lng" : 5.3363830            }         },         "icon" : "http://maps.gstatic.com/mapfiles/place_api/icons/lodging-71.png",         "id" : "cc30d6f4e6b0db1ed6dfd17bac118bbf78f194ae",         "name" : "Century (Hotel)",         "rating" : 3.80,         "reference" : "CnRsAAAAfw4ucOBllDjMCtRU9epYQelpKHRx3u8JR8jr4I5jFzJsktNn3Un-3LjKKWtXCk3Unofi_6RhPFw9Tp8VU_Z7i-WOa00P-pirtCDjQPxg7sp470Mgh6_6PQspGG39H3k-VIMuUz9HWmoP3sPjump6SRIQTp1LZghfi-_HVaMfQArGPxoU-6ogK2IGMiueMEXP9mFiHlGJmo8",         "types" : [ "lodging", "establishment" ]      },      {         "formatted_address" : "Maastrichterstraat 30, Hasselt, België",         "geometry" : {            "location" : {               "lat" : 50.9295410,               "lng" : 5.3391230            }         },         "icon" : "http://maps.gstatic.com/mapfiles/place_api/icons/cafe-71.png",         "id" : "352f9da4642588aba419353487b210baeb30ddd3",         "name" : "In De Kleine Hal",         "reference" : "CnRuAAAAAlnMpdiL4Kluiw1T0-QgKWFv9wwWvrSY4w-B7SLI7gvpqT0FYaMTl6DhxncPLqRyxOTWxw3DP3r1XvNHDKXnAb3eRM-gZYwIqbvzVq62zEZgGw0usQgf48mAWOrDpegdNLnEFRYkIduRwlDMDyIOyBIQlDs9BozH4m_gE-dZKZf7yxoUqw0H_HT_IQnLaVm0YwG1QV7o3kg",         "types" : [ "cafe", "food", "establishment" ]      },      {         "formatted_address" : "Fruitmarkt 32, Hasselt, België",         "geometry" : {            "location" : {               "lat" : 50.9299760,               "lng" : 5.3393550            }         },         "icon" : "http://maps.gstatic.com/mapfiles

11-03 14:47:13.864: D/URL INPUT(19532): https://maps.googleapis.com/maps/api/place/textsearch/json?query=cafe+in+Elfde-Liniestraat%2C+Hasselt%2C+Belgium&pagetoken=ClRHAAAAqYa7IpwMQH3q1DyinfsMnkuOaZg8wQrp0pvzarv13pwqruqT2YfaEry6v_d-L4d_TC13G6iHuPz-rc1KbI6XRE01rB5CuXcHISu-CI_CaKESEMUxSjHhHDNee4xbr2tPEZcaFPvO6xlRDjs-5AixyD9vamWwTGlU&sensor=true&key=<myKey>

11-03 14:47:13.989: D/URL OUTPUT(19532): {   "html_attributions" : [],   "results" : [],   "status" : "INVALID_REQUEST"}

PS: When I past the code in my browser (with next_page_token) both ways are correct, only I my Android app I get no valid data. AND yes I have a API Key, but I removed it here from both the code and my logcat output.

Anyone a idea what I'm doing wrong here?

EDIT When I hardcode my url (with next-page-token) in Android, and afterwords let it run. I get the data that I expected, so no Invalid_Request is returned. The problems lays somewhere in building the url in Android, or the usage of the URL-class.

Answer 1

Found the answer on my own question: after a while, I just needed to check my output of logcat better.

The times of the request en times when the data was sent back from google.

• output time of request 1: 14:47:13:832

• input time of request 2: 14:47:13:864

• output time of request 2: 14:47:13:989

These times are to close to each other, in my opinion Google thought I was a spammer/hacker or something and denied me access to the server on the second request.

The solution for me is wait 1 second if there is a next_page_token and then send the followup request.