Postman - how to loop request until I get a specific response?

Question

I'm testing API with Postman and I have a problem: My request goes to sort of middleware, so either I receive a full 1000+ line JSON, or I receive PENDING status and empty array of results:

{
  "meta": {
    "status": "PENDING",
    "missing_connectors_count": 0,
    "xxx_type": "INTERNATIONAL"
  },
  "results": []
}

The question is, how to loop this request in Postman until I will get status SUCCESS and results array > 0? When I'm sending those requests manually one-by-one it's ok, but when I'm running them through Collection Runner, "PENDING" messes up everything.

Answer 1

I found an awesome post about retrying a failed request by Christian Baumann which allowed me to find a suitable approach to the exact same problem of first polling the status of some operation and only when it's complete run the actual tests.

The code I'd end up if I were you is:

const maxNumberOfTries = 3; // your max number of tries
const sleepBetweenTries = 5000; // your interval between attempts

if (!pm.environment.get("tries")) {
    pm.environment.set("tries", 1);
}

const jsonData = pm.response.json();

if ((jsonData.meta.status !== "SUCCESS" && jsonData.results.length === 0) && (pm.environment.get("tries") < maxNumberOfTries)) {
     const tries = parseInt(pm.environment.get("tries"), 10);
     pm.environment.set("tries", tries + 1);
     setTimeout(function() {}, sleepBetweenTries);
     postman.setNextRequest(request.name);
 } else {
     pm.environment.unset("tries");

     // your actual tests go here...
}

What I liked about this approach is that the call postman.setNextRequest(request.name) doesn't have any hardcoded request names. The downside I see with this approach is that if you run such request as a part of the collection, it will be repeated a number of times, which might bloat your logs with unnecessary noise.

The alternative I was considering is writhing a Pre-request Script which will do polling (by sending a request) and spinning until the status is some kind of completion. The downside of this approach is the need for much more code for the same logic.

Answer 2

When waiting for services to be ready, or when polling for long-running job results, I see 4 basic options:

1. Use Postman collection runner or newman and set a per-step delay. This delay is inserted between every step in the collection. Two challenges here: it can be fragile unless you set the delay to a value the request duration will never exceed, AND, frequently, only a small number of steps need that delay and you are increasing total test run time, creating excessive build times for a common build server delaying other pending builds.
2. Use https://postman-echo.com/delay/10 where the last URI element is number of seconds to wait. This is simple and concise and can be inserted as a single step after the long running request. The challenge is if the request duration varies widely, you may get false failures because you didn't wait long enough.
3. Retry the same step until success with postman.setNextRequest(request.name);. The challenge here is that Postman will execute the request as fast as it can which can DDoS your service, get you black-listed (and cause false failures), and chew up a lot of CPU if run on a common build server - slowing other builds.
4. Use setTimeout() in a Pre-request Script. The only downside I see in this approach is that if you have several steps needing this logic, you end up with some cut & paste code that you need to keep in sync

Note: there are minor variations on these - like setting them on a collection, a collection folder, a step, etc.

I like option 4 because it provides the right level of granularity for most of my cases. Note that this appears to be the only way to "sleep" in a Postman script. Now standard javascript sleep methods like a Promise with async and await are not supported and using the sandbox's lodash _.delay(function() {}, delay, args[...]) does not keep script execution on the Pre-request script.

In Postman standalone app v6.0.10, set your step Pre-request script to:

console.log('Waiting for job completion in step "' + request.name + '"');

// Construct our request URL from environment variables
var url = request['url'].replace('{{host}}', postman.getEnvironmentVariable('host'));
var retryDelay = 1000;
var retryLimit = 3;

function isProcessingComplete(retryCount) {
    pm.sendRequest(url, function (err, response) {
        if(err) {
            // hmmm. Should I keep trying or fail this run? Just log it for now.
            console.log(err);
        } else {
            // I could also check for response.json().results.length > 0, but that
            // would omit SUCCESS with empty results which may be valid
            if(response.json().meta.status !== 'SUCCESS') {
                if (retryCount < retryLimit) {
                    console.log('Job is still PENDING. Retrying in ' + retryDelay + 'ms');
                    setTimeout(function() {
                        isProcessingComplete(++retryCount);
                    }, retryDelay);
                } else {
                    console.log('Retry limit reached, giving up.');
                    postman.setNextRequest(null);
                }
            }
        }
    });
}

isProcessingComplete(1);

And you can do your standard tests in the same step.

Note: Standard caveats apply to making retryLimit large.