I've written a script in vba to scrape some items from a webpage. By default, there are 4/5 items visible when I open that page. However, the webpage displays it's rest of the items when it is made to scroll downward. If it were not for split screen, I could have handled it using .parentWindow.scrollBy
. As I do not have any idea how to scroll any partial screen of a webpage, I get stuck. Any help on this will be highly appreciated.
Link to the webpage: Page_Link
This is what I have written so far:
Sub Get_Result()
Const URL As String = "replace with above link"
Dim IE As New InternetExplorer, html As HTMLDocument
Dim storage As Object, posts As Object, post As Object
With IE
.Visible = True
.navigate URL
Do Until .readyState = READYSTATE_COMPLETE: Loop
Set html = .document
End With
For i = 1 To 5
Set storage = html.getElementsByClassName("sc-jqCOkK gkztbk")
html.parentWindow.scrollBy 0, 100
'SendKeys "{end}"
Application.Wait Now + TimeValue("00:00:003")
Next i
For Each posts In storage
row = row + 1: Cells(row + 1, 1) = posts.querySelector("[id='app.components.HouseCard.rentLine']").innerText
Next posts
IE.Quit
End Sub
Btw, this is how that split window look like:
Post script: I'm only after any solution regarding scrolling a split screen like it is visible above.
Your website might not be scrolling if there is a problem with your browser (Google Chrome, Safari, etc.), website code, or your computer hardware. You can investigate & fix the issue by restarting the browser, looking at the site code, deleting cookies, extensions, and/or caches.
Scroll one page at a time in all major browsers including Microsoft Internet Explorer and Mozilla Firefox by pressing the Spacebar key. Move back up the page by pressing Shift + Spacebar or the Home key on the keyboard.
For the scrolling of the left window use the div
with class name sc-iqzUVk gZZIKO
and set scrollTop to some value, e.g. use scrollHeight
to scroll to the bottom like this leftWindowDiv.scrollTop = leftWindowDiv.scrollHeight
.
Complete example:
Dim leftWindowColl As IHTMLElementCollection
Set leftWindowColl = html.getElementsByClassName("sc-iqzUVk gZZIKO")
If leftWindowColl.Length > 0 Then
Dim leftWindowDiv As HTMLDivElement
Set leftWindowDiv = leftWindowColl.Item(0)
leftWindowDiv.scrollTop = leftWindowDiv.scrollHeight
End If
Because the class names seem to be generated (names like sc-iqzUVk gZZIKO
look like generated) it could be difficult to find the appropriate div
based on class name because the generated name will change. That is probably why you couldn't perform the scroll as you write in your comment. This can be solved by using the querySelector
method as already suggested by @Florent B. More about selectors.
This selector selectes all divs inside element with id app
which have style attribute ending with scroll;
.
Dim selector As String
selector = "#app div[style$='scroll;']"
Dim leftWindowDiv As HTMLDivElement
Set leftWindowDiv = html.querySelector(selector)
leftWindowDiv.scrollTop = leftWindowDiv.scrollHeight
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With