Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to read multiple Sets stored on Redis using some command or LUA script

I want to get all the sets from Redis using a list of keys in single call. As per the documentation, Redis provides SSCAN command for that but as I am using StackExchange.Redis as a Redis adapter, I guess this command does not have any such method in this adapter. So there two things I am looking for:

  • I looking forward to execute SSCAN using the LUA script but was not able to find any such example over the internet. Can anyone share how to call SSCAN from LUA with multiple SET Keys.
  • Also for the StackExchange.Redis, if I execute multiple SetMembers() in a transaction, is it similar to use SSCAN() command using LUA script?

Thanks

like image 955
Rupendra Avatar asked Apr 12 '17 03:04

Rupendra


People also ask

Which Lua command causes script to terminate when Redis returns?

call() will raise a Lua error that in turn will force EVAL to return an error to the command caller, while redis. pcall will trap the error returning a Lua table representing the error." So according to the documentation , in the case of using redis.

Is Lua script Atomic?

Lua scripts are executed atomically, that is, no other script or command will run while a script is running, which gives us the same transactional semantics as MULTI / EXEC .

How do I view the contents of Redis cache?

You can check the actual number by running redis-cli config get databases. In interactive mode, the database number is displayed in the prompt within square braces. For example, 127.0. 0.1:6379[13] shows that the 13th database is in use.

What is a Lua script in Redis?

Executing Lua in Redis. Redis lets users upload and execute Lua scripts on the server. Scripts can employ programmatic control structures and use most of the commands while executing to access the database. Because scripts execute in the server, reading and writing data from scripts is very efficient.


2 Answers

The sample C# code to get many SETs in a single call is as following: I am using StackExchange.Redis as a Redis connector:

using StackExchange.Redis;
using System;
using System.Text;

namespace RedisGetMultipleKeys
{
/// <summary>
/// Class to perofrme operations on SE.Redis
/// </summary>
class Program
{
    /// <summary>
    /// Executes necessary pre-requisites 
    /// </summary>
    /// <param name="args"></param>
    static void Main(string[] args)
    {

        //Connect Redis
        var _cache = Program.Connect();

        //Store 10k Sets
        string prefix = "user";
        StringBuilder keys = new StringBuilder();
        for (int i = 0; i < 10000; i++)
        {
            keys.Append(" " + prefix + i);
            _cache.SetAdd(prefix + i, i);
        }

        var keyList = new RedisKey[10000];
        //Generate keys array
        for (int i = 0; i < 10000; i++)
        {
            var key = new RedisKey();
            key = prefix + i;
            keyList.SetValue(key, i);
        }

        var startTime = DateTime.Now;
        //Perform SUNION
        var values = _cache.SetCombine(SetOperation.Union, keyList);

        var endTime = DateTime.Now;
        TimeSpan diff = endTime.Subtract(startTime);

        Console.WriteLine("total time taken to read 10k keys = " + diff);
        Console.Read();

        //TODO: to be changed accordingly to read Set values returned other than String
        foreach (var value in values)
        {
            Console.WriteLine(value.ToString());
        }

        endTime = DateTime.Now;
        diff = endTime.Subtract(startTime);

        Console.WriteLine("total time taken to read 10k keys = " + diff);
        Console.Read();

    }


    /// <summary>
    /// Connects to Redis db
    /// </summary>
    /// <returns>Returns an instance of Redis db</returns>
    private static IDatabase Connect()
    {
        string redisConnection = "localhost:6379,ssl=false,allowAdmin=true,ConnectRetry=3,ConnectTimeout=5000,defaultDatabase=1";
        ConnectionMultiplexer connection = ConnectionMultiplexer.Connect(redisConnection);
        return connection.GetDatabase();
    }
}

}

I hope it will help the C# developers looking for the solution. Thanks to Mgravell from SE.Redis dev team for helping me by his suggestions. More discussion could be found here at GitHub How to get multiple sets by passing set key list in a single call

like image 130
Rupendra Avatar answered Sep 24 '22 12:09

Rupendra


get multiple sets in a single call by passing list of Set keys

I'm unclear about the specific requirements here, so there are probably several ways to do that. The simplest is by calling SUNION, which will dedupe and return the results w/o order.

Another option is to use a Lua script, such as:

local reply = {}
for i = 1,#KEYS do
  local elems = redis.call('SMEMBERS', KEYS[i])
  table.insert(reply, elems)
end
return reply

redis-cli example (sorry, .NET isn't my forte):

$ redis-cli SADD s1 foo bar
(integer) 2
$ redis-cli SADD s2 baz qaz
(integer) 2
$ redis-cli --eval script.lua s1 s2
1) 1) "foo"
   2) "bar"
2) 1) "baz"
   2) "qaz"

Note: if your Sets have a lot of members then getting all of them, regardless the approach, is going to be "expensive" - reconsider the need for that.

like image 31
Itamar Haber Avatar answered Sep 26 '22 12:09

Itamar Haber