VBA Internet Explorer wait for web page to load

Question

I know questions like this have been asked before, but mine is a bit different and has been fairly troubling. What I'm dealing with is a web page with a form with a few events that load more of the page when certain items in input boxes are filled out. When these events fire the page loads again, but remains at the same URL with the same nameprop. I've been using the following types of methods both seperately and strung together to handle waiting for the page to load, but sometimes the VBA still manages to continue executing and set the HTMLDocument variable to a page without the appropriate information on it causing the macro to debug. Here are the kinds of things I've been trying so far:

While IE.Busy
    DoEvents
Wend

Do Until IE.statusText = "Done"
    DoEvents
Loop

Do Until IE.readyState = 4
    DoEvents
Loop

I've even attempted to place these events into a loop like the following, but it didn't quite work because the lastModified property only returns a value down to the second and the macro spins through the fields fast enough that it is returning a new page in the same second:

Do Until IE.statusText = "Done" And IE.Busy = False And IE.ReadyState = 4 _
And IE.document.lastModified > LastModified ----or---- IE.document.nameprop = _
"some known and expected name prop here"
    While IE.Busy
        DoEvents
    Wend
    Do Until IE.statusText = "Done"
        DoEvents
    Loop
    Do Until IE.readyState = 4
        DoEvents
    Loop
Loop

Even that fails to wait long enough to set the HTMLDocument object leading to a debug. I've contemplated setting the next input element and checking that for nothing to further the code, but even that wouldn't be successful 100% of the time because generally the Input elements exist in the HTML but are hidden until the appropriate event is fired, which wouldn't be a problem but they don't load their possible selections until after the event is fired. It might be an odd page.

Anyway... not sure what else to add. If there is something else that might be helpful to see just ask. I guess what I'm looking for is a way to get VBA to wait until IE knows another page isn't on it's way. It seems to load a few times before it is completely done.

So... Anyone have any ideas?

EDIT: Found a few new things to try. Still, no dice. It was suggested that I add these attempts. Here is the code, for some reason the VBE and excel instance become non-responsive when using this approach after firing an event that should populate the options on the select element... thinking about trying xml... here is the code:

intCounter = 0
Do until intCounter > 2
    Do Until IE.Busy = False: DoEvents: Loop
    Do Until IE.ReadyState = 4: DoEvents: Loop
    Set HTMLDoc = IE.Document
    Do Until HTMLDoc.ReadyState = "complete"
    Set HTMLSelect = HTMLDoc.getElementById("ctl00$ctl00$MainContent$ChildMainContent$ddlEmployeeBranchCodes")
    intCounter = 0
    For each Opt in HTMLSelect
        intCounter = intCounter + 1
    Next Opt
Loop

Based on what I can see happening on the web page, I know that it is somewhere in this loop that the VBE and Excel become non-responsive.

Hope that helps... I know it didn't help me... Drats.

EDIT: Just thought I'd add this. When it comes to automating a web page, for the most part, I no longer use IE. I've found it's much better, and sidesteps this issue of async stuff entirely, to simply perform the posts and gets yourself. May not be the best solution depending on what you're trying to do, but it works pretty reliably if you look at the traffic closely and parameterize things well.

Answer 1

After an exhaustive search, I've determined that the AJAX request, javascript code that runs asynchronously in the background isn't something I can get a signal from in any way. It does seem to trigger some event when it finishes with loading the page, which is an option I'd like to explore in the future. However, for my purposes I simply used the same code I was already using to fill out the form on my page and I have it loop through each field again to check to see if the values are still correct before clicking the submit button. It isn't an ideal solution, but it is a workaround that appears to have taken care of my issue in this case.

I'm not sure if my solution would be applicable to someone else dealing with this issue, but there it is if it helps. I think workarounds for this issue are going to have to be based on the application and web page in question.

Answer 2

Old question I know, but I think this is a good answer that I haven't seen about much...

---

I had a similar problem; waiting for all the images on a google image search to load (which is a tricky thing since image loads are prompted by AJAX and the user scrolling the page). As has been suggested in the comments; my solution was to wait for a certain element to appear in the viewport (this is an area a little larger than the monitor screen, and is treated as what you can actually "see").

This is achieved with the getBoundingClientRect method

Dim myDiv As HTMLDivElement: Set myDiv = currPage.getElementById("fbar") 
'myDiv should some element in the page which will only be visible once everything else is loaded
Dim elemRect As IHTMLRect: Set elemRect = myDiv.getBoundingClientRect
Do Until elemRect.bottom > 0 'If the element is not in the viewport, then this returns 0
    DoEvents
    'Now run the code that triggers the Ajax requests
    'For me that was simply scrolling down by a big number
    Set elemRect = myDiv.getBoundingClientRect
Loop
myDiv.ScrollIntoView

I explain in detail in the linked answer how this works, but essentially the BoundingClientRect.bottom is equal to 0 until the element is in the vieport.

The element is something which is loaded straight away (like a frame/template for the page). But you don't actually see it until all the content has been loaded, because it's right at the bottom.

If that element is indeed the last thing to be loaded (for my google search it was the Show More Results button), then as long as you get it into the viewport when it's loaded, you should be able to detect when it appears on the page. .bottom then returns a non-zero value (something to do with the actual position of the element on the page - I didn't really care though for my purposes). I finish off with a .ScrollIntoView, but that's not essential.

Answer 3

I had the same problem with My Webpage.. What is did is...

the fist option is

While Ie.**document**.readystate="complete"
DoEvents
Wend

there were few boxes in which options were loaded after a button click/even fire in another box...I jsst placed code like this..

Do Until IE.document.getelementbyid("Next box").Lenght>0
DoEvents
Loop
Application.wait Now+Timevalue("00:00:02)