Using Redis SCAN in NODE

Question

I have Redis with a lot of keys in some format and I want to get keys that match some pattern and do some operations on them. I don't use KEYS method since it's not recommend in production. Using SCAN I'm wondering what is the best way to write it in code. I have to do something like a while loop but using promises, my current solution looks like this (code is simplified a little):

'use strict'
const Promise = require('bluebird');
const config = require('./config');
const client = require('./clinet');

let iterator = 0;
Promise.coroutine(function* () {
  do {
    iterator = yield clinet.scanAsync(iterator, 'myQuery', 'COUNT', config.scanChunkSize)
      .then(data => {
        let nextIterator = data[0];
        let values = data[1];
        //do some magic with values
        return nextIterator;
      })
  } while (iterator !== '0');
})();

Is there a better way to do it that I'm missing?

Answer 1

Nice option for node-redis module is to use scan iterators. Example:

const redis = require("redis");
const client = redis.createClient();

async function getKeys(pattern="*", count=10) {
    const results = [];
    const iteratorParams = {
        MATCH: pattern,
        COUNT: count
    }
    for await (const key of client.scanIterator(iteratorParams)) {
        results.push(key);
    }
    return results;
}

(Of course you can also process your keys on the fly in for await loop without storing them in additional array if that's enough for you).

If you do not want to override scan parameters (MATCH/COUNT) you can just skip them and execute client.scanIterator() without parameter (defaults will be used then, MATCH="*", COUNT=10).

Answer 2

I realize this is a really old question, but I found all of the other answers very unsatisfying. Here is yet another attempt to scan in a relatively clean way using async await (WITHOUT the use of yet another external dependency). You can easily modify this to continuously delete each set of found keys (you would want to tackle them in batches like this in case there are LOTS). Pushing them into an array just demonstrates one very basic thing you could do with them during this stage.

const redis = require('redis');
const { promisify } = require('util');

const client = redis.createClient({...opts});
const scan = promisify(client.scan).bind(client);

const scanAll = async (pattern) => {
  const found = [];
  let cursor = '0';

  do {
    const reply = await scan(cursor, 'MATCH', pattern);

    cursor = reply[0];
    found.push(...reply[1]);
  } while (cursor !== '0');

  return found;
}