How to adjust the number of results returned when querying the Stackoverflow API?

Question

I'm using the Stackoverflow JSON API to retrieve questions marked with a given tag.

I have this small program in Java which retrieves questions marked with the "Java" tag.

public static void main(String[] args) throws Exception
{
    String urlString = "https://api.stackexchange.com/2.1/questions?order=desc&sort=votes&tagged=java&site=stackoverflow";

    URL url = new URL( urlString );

    BufferedReader reader = null;

    StringBuffer buffer = new StringBuffer();
    try
    {
        URLConnection connection = url.openConnection();

        InputStream isConn = connection.getInputStream();

        reader = new BufferedReader( new InputStreamReader( new GZIPInputStream( isConn ) ) );

        String inputLine;

        while (( inputLine = reader.readLine() ) != null)
        {
            buffer.append( inputLine );
        }
    }
    finally
    {
        if (reader != null)
        {
            reader.close();
        }
    }

    JSONObject jsonObject = new JSONObject( buffer.toString() );

    JSONArray jsonArray = jsonObject.getJSONArray( "items" );

    System.out.println( buffer );
    System.out.println( jsonArray.length() );
}

My problem is that it returns only 30 questions. Since my goal is to build a dataset for further textual analysis, I need to access way more than just 30 questions.

Is there a way to adjust the size of the returned list?

If not, how can I workaround this situation?

Answer 1

Notice the has_more property in the returned JSON, this indicates that more results are available. You can page through these results using the page and pagesize parameters in the url. The issue I foresee is the code will be pulling a large number of questions considering it will iterate through all java questions, so you may want to add a conditional that stops at a certain number of pages. Here is a quick example:

public static void main(String[] args) throws Exception {

    BufferedReader reader = null;
    int page = 1;
    JSONObject jsonObject = null;
    try {
        while (jsonObject == null || jsonObject.getBoolean("has_more")) {
            String urlString = "https://api.stackexchange.com/2.1/questions?order=desc&sort=votes&tagged=java&site=stackoverflow&pagesize=100";
            urlString += "&page=" + page++;
            URL url = new URL(urlString);
            URLConnection connection = url.openConnection();

            InputStream isConn = connection.getInputStream();
            StringBuffer buffer = new StringBuffer();
            reader = new BufferedReader(new InputStreamReader(new GZIPInputStream(isConn)));

            String inputLine;

            while ((inputLine = reader.readLine()) != null) {
                buffer.append(inputLine);
            }

            jsonObject = new JSONObject(buffer.toString());
            JSONArray jsonArray = jsonObject.getJSONArray("items");

            System.out.println(buffer);
            System.out.println(jsonArray.length());
        }
    } finally {
        if (reader != null) {
            reader.close();
        }
    }

}