Menu
How to disable images/CSS in Puppeteer? I've seen this tutorial https://www.scrapehero.com/how-to-increase-web-scraping-speed-using-puppeteer/ but I don't know how to translate it to Python
599
For instance, when you avoid loading images, the screenshots won’t appear as you imagined. Puppeteer only works with Chrome and Chromium. For automating other browsers you might want to try the Selenium framework.
Since Puppeteer gives full control over the Chrome browser, we can provide an interceptor on every request and cancel the ones we don’t require. For scraping, we don’t really need to worry about any visuals, including the images so we will check each request made by Chrome and block the ones with images and CSS resources.
Puppeteer allows blocking any outgoing requests while loading the page. Whether you want to block ads, tracking scripts, or different types of resources, it is relatively easy to do with Puppeteer. If you want to speed up scrapping or make screenshots faster, you can disable all the requests that do not make any crucial impact on the results.
To execute the puppeteer script save the code inside the directory created and run the script as The general idea is to not let the headless browser run any command that doesn’t help with the scraping. This includes loading images, CSS and fonts.
Based on example from https://github.com/miyakogi/pyppeteer/blob/dev/pyppeteer/page.py#L312:
await page.setRequestInterception(True)
async def intercept(request):
if any(request.resourceType == _ for _ in ('stylesheet', 'image', 'font')):
await request.abort()
else:
await request.continue_()
page.on('request', lambda req: asyncio.ensure_future(intercept(req)))
This below code will disable resource by type: fetch, image, media, and font.
page.setRequestInterception(true)
page.on ( 'request', async request => {
if ( request.resourceType () === 'fetch' || request.resourceType () === 'image' || request.resourceType () === 'media' || request.resourceType () === 'font' ) {
request.abort ()
} else {
request.continue ()
}
})
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
