Selenium extremely slow on reading the DOM

Question

Selenium interaction with DOM seems extremely slow while doing couple of things in every page instantiation. Throughout the site we have visible spinner that indicates any outstanding API calls resolved or not. In summary I have three methods that make sure the stability of page before performing any action.

1. Check for the DOM ready state
2. Check for any outstanding JQuery calls
3. Check for loading spinners

All of these three are done as a part of the page object instantiation with following methods.

    public static void waitForLoadingAllSpinnersAnywhere(final WebDriver driver){
    final WebDriverWait wait = new WebDriverWait(driver, timeout);
    
    wait.until(waitForDomReadyState());
    wait.until(waitForjQueryToBeInactive());
    List<WebElement> elements = wait.until(ExpectedConditions.presenceOfAllElementsLocatedBy(spinnersLoacator));
    
    for(WebElement element: elements){
        wait.until(invisibilityOfElementLocated(element));  
     }
    }

    private static ExpectedCondition<Boolean> waitForDomReadyState(){

        return new ExpectedCondition<Boolean>() {

            @Override
            public Boolean apply(WebDriver d){

                return ( ((JavascriptExecutor) d).executeScript("return document.readyState;").equals("complete"));
            }
        };
    }


    private static ExpectedCondition<Boolean> waitForjQueryToBeInactive(){

        return new ExpectedCondition<Boolean>() {

            @Override
            public Boolean apply(WebDriver d){

                return (Boolean) ( ((JavascriptExecutor) d).executeScript("return jQuery.active == 0;"));
            }
        };
    }

    public static ExpectedCondition<Boolean> invisibilityOfElementLocated(final WebElement element){

        return new ExpectedCondition<Boolean>() {

            @Override
            public Boolean apply(WebDriver driver){

                try{
                    return !element.isDisplayed();
                } catch (NoSuchElementException | StaleElementReferenceException e){
                    // Returns true because the element is not present in DOM.
                    // The
                    // try block checks if the element is present but is
                    // invisible or stale
                    return true;
                }
            }
        };
    }

Taking an example of a page(say patient page) which has good number of API calls and fetches a lot of data. For a initial class instantiation it takes about 17s(log below). My Selenium knowledge says, the subsequent page instantiation should not take same or more time to check DOM ready state, or JQuery call or spinner waits since there is nothing changing at all. However, every time new page instantiate I see it takes same amount of time taken to check all these three. What's happening there? Does Selenium actually tries to interact with Server every time I do these or just interaction with the client is slow for some reason? If so, what could be the possible answer?

Console log

==== [[Finished waiting for 8 spinner elements found on widget [Patient] after [17] s]]

==== [[Start waiting for 8 spinner elements found on widget [Patient] ]]

==== [[Finished waiting for 8 spinner elements found on widget [Patient] after [17] s]]

==== Browser on [[[Patient]]]

==== [[Start waiting for 8 spinner elements found on widget [Patient] ]]

==== [[Finished waiting for 8 spinner elements found on widget [Patient] after [17] s]]

Environment:

1. Selenium 2.48
2. Firefox 38

I also tried with Selenium 2.52 and firefox 44 with same result

Answer 1

Selenium handles all the waiting on the client side with a request sent to the server for every evaluation until the condition is met. It can quickly degenerate in case of high latency, especially if there are a lot of calls. Moreover some evaluations require a script injection which doesn't help either.

So the best way to improve the performance in your case would be to use a single asynchronous JavaScript call:

public static void waitForLoadingAllSpinnersAnywhere(final WebDriver driver) {
  const String JS_WAIT_SPINNERS = 
      "var callback = arguments[0]; " +
      "(function fn(){ " +
      "  if (document.readyState == 'complete' && jQuery.active == 0) { " +
      "    var elts = $('.spinners'); " +
      "    if (elts.length == 8 && !elts.is(':visible')) " +
      "      return callback(); " +
      "  } " +
      "  setTimeout(fn, 60); " +
      "})();";

   ((JavascriptExecutor)driver).executeAsyncScript(JS_WAIT_SPINNERS);
}

To initialize the timeout:

driver.manage().timeouts().setScriptTimeout(30, TimeUnit.SECONDS);