How to programmatically replicate a request found in Chrome Developer Tools?

Question

I'm looking at my balance on Venmo.com but they only show you 3 months at a time and I'd like to get my entire transaction history.

Looking at the Chrome Developer Tools, under the network tab, I can see the request to https://api.venmo.com/v1/transaction-history?start_date=2017-01-01&end_date=2017-01-31 which returns JSON.

I'd like to programmatically iterate through time and make several request and aggregate all of the transactions. However, I keep getting 401 Unauthorized.

My initial approach was just using Node.js. I looked at the cookie in the request and copied it into a secret.txt file and then sent the request:

import fetch from 'node-fetch' import fs from 'fs-promise'  async function main() {   try {     const cookie = await fs.readFile('secret.txt')       const options = {       headers: {         'Cookie': cookie,       },      }     try {       const response = await fetch('https://api.venmo.com/v1/transaction-history?start_date=2016-11-08&end_date=2017-02-08', options)       console.log(response)     } catch(e) {       console.error(e)     }   } catch(e) {     console.error('please put your cookie in a file called `secret.txt`')     return   } } 

That didn't work do I tried copying all of the headers over:

const cookie = await fs.readFile('secret.txt')   const options = {   headers: {     'Accept-Encoding': 'gzip, deflate, sdch, br',     'Accept-Language': 'en-US,en;q=0.8',     'Cache-Control': 'no-cache',     'Connection': 'keep-alive',     'Cookie': cookie,     'Host': 'api.venmo.com',     'Origin': 'https://venmo.com',     'Pragma': 'no-cache',     'Referer': 'https://venmo.com/account/settings/balance/statement?end=02-08-2017&start=11-08-2016',     'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36',   },  } try {   const response = await fetch('https://api.venmo.com/v1/transaction-history?start_date=2016-11-08&end_date=2017-02-08', options)   console.log(response) } catch(e) {   console.error(e) } 

This also did not work.

I even tried making the request from the console of the website and got a 401:

fetch('https://api.venmo.com/v1/transaction-history?start_date=2016-11-08&end_date=2017-02-08', {credentials: 'same-origin'}).then(console.log) 

So my question here is this: I see a network request in Chrome Developer Tools. How can I make that same request programmatically? Preferably in Node.js or Python so I can write an automated script.

Answer 1

In the Network tab of the Chrome Developer Tools, right click the request and click "Copy" > "Copy as cURL (bash)". You can then either write a script using the curl command directly, or use https://curlconverter.com/ to convert the cURL command to Python, JavaScript, PHP, R, Go, Rust, Elixir, Java, MATLAB, Dart or JSON.