Wait until page is loaded before reading contents from a URL in Java

Question

I am reading from a url in my java code but the page I want to read executes a command when loaded and the InputStreamReader reads the page before it has completely loaded, so my buffered reader only collects the HTML on the page before the real content is loaded.

My main goal is to find the word "sales" on the page, but I can't do this if the stream opened is connected before the full page is loaded. Is there a way to wait for it to load or something?

Here is my code:

URL url = new URL("http://urlgoeshere.com?"+ withAParam);
        URLConnection uc = url.openConnection();
        uc.connect();
        BufferedReader in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
        String inputLine = in.readLine();
        int index = -1;             
        while ((inputLine = in.readLine()) != null){
            index=inputLine.toLowerCase().indexOf("sales");
            if(index>=0){
            log.info("Found sales!");
                break;                  
            }
        }
        if (in != null){
            in.close(); 
        }

Answer 1

Now first some Java coding tips that won't solve your problem then a tip that may.

You should refactor your code and use try-finally where you close the stream in finally block to make sure it always closes even when an exception is thrown. Then I wouldn't use the indexOf with an int. To make the code more sharp,readable and less verbose write if(inputLine.toLowerCase().contains("sales")){ directly in your if statement and remove all index code.

You can try the apache API http://hc.apache.org/httpcomponents-client-ga/index.html to fetch the homepage.