I'm trying to send a request to this endpoint: https://9gag.com/v1/group-posts/group/default/type/trending
to get 9gag posts data.
Works fine on postman and chrome, but when I tried using axios
from Node.js, it responded with 403
(and reading the returned DOM, it believe that it's asking me to enter captcha), and when I tried to send a request using node-fetch
, it responded with 200
with posts data (which is the correct one).
Question is, how come node-fetch
works fine but axios
doesn't?
Here's the code I used for testing:
import axios from "axios";
import fetch from "node-fetch";
const URL = "https://9gag.com/v1/group-posts/group/default/type/trending";
// Inside async function
await fetch(URL); // Responded with 200 and json data
await axios(URL); // Responded with 403 and HTML DOM, axios.get() also gives the same result
If you take a look at https://9gag.com/robots.txt, you might notice Mediapartners-Google. It's a User-Agent you have to specify in headers to bypass the Cloudflare captcha.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With