Scrape data with cheerio and nodejs

Question

I used cheerio module to scrape list of products from a website. But have problem, website using "Infinite Scrolling" to load data (data load more when scroll down). So, cheerio can't get all products.

Can someone suggest me a solutions for this problem ? Thanks you.

Answer 1

You can check out what sort of queries the page performs when you scroll. Then just hit those queries yourself and get the data.

Alternatively you can try to use something like node-phantom-simple, trigger scroll via Phantom and scrape.

I would go with the first option myself.