How to get the number of <p> tags inside div in scrapy?

Question

I am scraping this website link.

The last of the <p> tags contains the user_info and it is creating problem for me as I am using -

''.join(response.xpath('//div[@class="entry-content"]/p[2]/text()').extract())

But p[2] changes if the text above it is in good numbers. Say here it is p[5]

I am thinking of this to calculate the number of <p>tags inside the div and assign the number to my item

How to deal with this problem?

Answer 1

From what I understand, this is just the last paragraph in the entry content - you can use last():

//div[@class="entry-content"]/p[last()]/text()

Works for me.

Answer 2

If you just want to count the p elements

len(response.xpath('//div[@class="entry-content"]/p'))