Menu
I want to get the JSON data from a website I'm scraping with Puppeteer, but I can't figure how to get the body of the request back. Here's what I've tried:
const puppeteer = require('puppeteer')
const results = [];
(async () => {
const browser = await puppeteer.launch({
headless: false
})
const page = await browser.newPage()
await page.goto("https://capuk.org/i-want-help/courses/cap-money-course/introduction", {
waitUntil: 'networkidle2'
});
await page.type('#search-form > input[type="text"]', 'bd14ew')
await page.click('#search-form > input[type="submit"]')
await page.on('response', response => {
if (response.url() == "https://capuk.org/ajax_search/capmoneycourses"){
console.log('XHR response received');
console.log(response.json());
}
});
})()
This just returns a promise pending function. Any help would be great.
672
Because the xhr.responseText is a string, you can easily manipulate it. To work with the data, you need to convert it back into an object. You can do this using the JSON.parse () method. Here it is in full context again.
Inside the if block, parse the response object from the request to a JSON to get the JSON of your Mongo collection and assign it to your state variable. This is how you make an XHR request to get JSON of your Mongo collection and store it inside a local state of your component.
Making an XHR Request Inside your React app, import useState and useEffect. The useState hook is used to create a state where the JSON object from the endpoint is stored. Make an XHR request inside the useEffect lifecycle hook's callback function using the XMLHttpRequest object.
Make an XHR request inside the useEffect lifecycle hook's callback function using the XMLHttpRequest object. Have a look at the following code: Inside the useEffect ()' s callback function, create a new instance of the XMLHttpRequest object.
As response.json returns a promise we need to await it.
page.on('response', async (response) => {
if (response.url() == "https://capuk.org/ajax_search/capmoneycourses"){
console.log('XHR response received');
console.log(await response.json());
}
});
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
