Loop over a set of pages with Selenium Basic (VBA)

Question

Task:

So my first foray into Selenium and I am attempting to:

1. Find the number of pages in a pagination set listed at the bottom of https://codingislove.com/ This is purely to support task 2 by determining the loop end.
2. Loop over them

I believe these are linked but for those that want a single issue. I simply want to find the correct collection and loop over it to load each page.

The number of pages is, at time of writing, 6 as seen at the bottom of the webpage and shown below:

As an MCVE I simply want to find the number of pages and click my way through them. Using Selenium Basic.

What I have tried:

I have read through a number of online resources, I have listed but a few in references.

Task 1)

It seems that I should be able to find the count of pages using the Size property. But I can't seem to find the right object to use this with. I have made a number of attempts; a few shown below:

bot.FindElementsByXPath("//*[@id=""main""]/nav/div/a[3]").Size '<==this I think is too specific
bot.FindElementsByClass("page-numbers").Size

But these yield the run-time error 438:

"Object does not support this property or method"

And the following doesn't seem to expose the required methods:

bot.FindElementByCss(".navigation.pagination")

I have fudged with

bot.FindElementsByClass("page-numbers").Count + 1 

But would like something more robust

Task 2)

I know that I can navigate to the next page, from page 1, with:

bot.FindElementByXPath("//*[@id=""main""]/nav/div/a[3]").Click

But I can't use this in a loop presumably because the XPath needs to be updated. If not updated it leads to a runtime error 13.

As the re-directs follow a general pattern of

href="https://codingislove.com/page/pageNumber/"

I can again fudge my way through by constructing each URL in the loop with

bot.Get "https://codingislove.com/page/" & i & "/"

But I would like something more robust.

Question:

How do I loop over the pagination set in a robust fashion using selenium? Sure I am having a dense day and that there should be an easy to target appropriate collection to loop over.

Code - My current attempt

Option Explicit
Public Sub scrapeCIL()
    Dim bot As New WebDriver, i As Long, pageCount As Long

    bot.Start "chrome", "https://codingislove.com"
    bot.Get "/"
    pageCount = bot.FindElementsByClass("page-numbers").Count + 1 '

    For i = 1 To pageCount 'technically can loop from 2 I know!
      ' bot.FindElementByXPath("//*[@id=""main""]/nav/div/a[3]").Click 'runtime error 13
       ' bot.FindElementByXPath("//*[@id=""main""]/nav/div/a[2]/span").Click ''runtime error 13
        bot.Get "https://codingislove.com/page/" & i & "/"
    Next i

    Stop

    bot.Quit
End Sub

Note:

Any supported browser will do. It doesn't have to be Chrome.

References:

1. Finding the number of pagination buttons in Selenium WebDriver
2. http://seleniumhome.blogspot.co.uk/2013/07/how-can-we-automate-pagination-using.html

Requirements:

1. Selenium Basic
2. ChromeDriver 2.37 'Or use IE but zoom must be at 100%
3. VBE Tools > references > Selenium type library

Answer 1

To click the element, it must be visible in the screen, so you need to scroll to the bottom of the page first (selenium might do this implicitly some times, but I don't find it reliable).

Try this:

Option Explicit
Public Sub scrapeCIL()
    Dim bot As New WebDriver, btn As Object, i As Long, pageCount As Long

    bot.Start "chrome", "https://codingislove.com"
    bot.Get "/"
    pageCount = bot.FindElementsByClass("page-numbers").Count

    For i = 1 To pageCount

        bot.ExecuteScript ("window.scrollTo(0,document.body.scrollHeight);")

        Application.wait Now + TimeValue("00:00:02")

        On Error Resume Next
        Set btn = bot.FindElementByCss("a[class='next page-numbers']")
        If btn.IsPresent = True Then
            btn.Click
        End If
        On Error GoTo 0

    Next i

    bot.Quit

End Sub