Triggering parallel of 1k HTTP requests would get stuck

Question

The question is what is actually happening when you trigger 1k-2k outgoing HTTP requests? I see that it would resolve all the connections easily with 500 connections but moving up towards from there seems to cause problems as the connections are left open and Node app would be stuck there. Tested with local server + example Google and other mock servers.

So with some different server endpoints I did receive reason: read ECONNRESET which is fine the server couldn't handle the request and throw an error. In 1k-2k request range the program would just hang. When you check the open connections with lsof -r 2 -i -a you could see that there are some X amount of connections that keep hanging there 0t0 TCP 192.168.0.20:54831->lk-in-f100.1e100.net:https (ESTABLISHED). When you add timeout setting to requests these would probably end up with timeout error, but why otherwise the connection is kept up forever and the main program would end up in some limbo state?

Example code:

import fetch from 'node-fetch';

(async () => {
  const promises = Array(1000).fill(1).map(async (_value, index) => {
    const url = 'https://google.com';
    const response = await fetch(url, {
      // timeout: 15e3,
      // headers: { Connection: 'keep-alive' }
    });
    if (response.statusText !== 'OK') {
      console.log('No ok received', index);
    }
    return response;
  })

  try {
    await Promise.all(promises);
  } catch (e) {
    console.error(e);
  }
  console.log('Done');
})();

Answer 1

To understand what was happening for sure, I needed to make some modifications to your script, but here there are.

First, you might know how node and its event loop works, but let me make a quick recap. When you run a script, node runtime first run the synchronous part of it then schedule the promises and timers to be executed on the next loops, and when checked they are resolved, run the callbacks in another loop. This simple gist explains it very well, credit to @StephenGrider:

const pendingTimers = [];
const pendingOSTasks = [];
const pendingOperations = [];

// New timers, tasks, operations are recorded from myFile running
myFile.runContents();

function shouldContinue() {
  // Check one: Any pending setTimeout, setInterval, setImmediate?
  // Check two: Any pending OS tasks? (Like server listening to port)
  // Check three: Any pending long running operations? (Like fs module)
  return (
    pendingTimers.length || pendingOSTasks.length || pendingOperations.length
  );
}

// Entire body executes in one 'tick'
while (shouldContinue()) {
  // 1) Node looks at pendingTimers and sees if any functions
  // are ready to be called.  setTimeout, setInterval
  // 2) Node looks at pendingOSTasks and pendingOperations
  // and calls relevant callbacks
  // 3) Pause execution. Continue when...
  //  - a new pendingOSTask is done
  //  - a new pendingOperation is done
  //  - a timer is about to complete
  // 4) Look at pendingTimers. Call any setImmediate
  // 5) Handle any 'close' events
}

// exit back to terminal

Note that the event loop will never end until there is pending OS tasks. In other words, your node execution will never end until there is pending HTTP requests.

In your case, it runs an async function, as it will always return a promise, it will schedule it to be executed in the next loop iteration. On your async function, you schedule another 1000 promises (HTTP requests) at once in that map iteration. After that, you are awaiting all of then be resolved to finish the program. It will work, for sure, unless your anonymous arrow function on the map doesn't throw any error. If one of your promises throws an error and you don't handle it, some of the promises won't have their callback called ever making the program to end but not to exit, because the event loop will prevent it to exit until it resolves all the tasks, even without callback. As it says on the Promise.all docs: it will reject as soon as the first promise rejects.

So, your on ECONNRESET error is not related to node itself, is something with your network that made the fetch to throw an error and then prevent the event loop to ends. With this little fix, you would be able to see all the requests being resolved asynchronously:

const fetch = require("node-fetch");

(async () => {
  try {
    const promises = Array(1000)
      .fill(1)
      .map(async (_value, index) => {
        try {
          const url = "https://google.com/";
          const response = await fetch(url);
          console.log(index, response.statusText);
          return response;
        } catch (e) {
          console.error(index, e.message);
        }
      });
    await Promise.all(promises);
  } catch (e) {
    console.error(e);
  } finally {
    console.log("Done");
  }
})();