Menu
I was trying to put some heavy load on my Redis for testing purposes and find out any upper limits. First I loaded it with 50,000 and 100,000 keys of size 32characters with values around 32 characters. It took no more than 8-15 seconds in both key sizes. Now I try to put 4kb of data as value for each key. First 10000 keys take 800 milli seconds to set. But from that point it slows down gradually and to set whole 50,000 keys it takes aroudn 40 minutes. I am loading the database using NodeJs with node_redis (Mranney) . Is there any mistake I am doing or is Redis just that slow with big values of size 4 KB?
One more thing I found now is when I run another client parallel to the current one and update keys this 2nd client finishes up loading the 50000 keys with 4kb values within 8 seconds while the first client still does its thing forever. Is it a bug in node or the redis library? This is alarming and not acceptable for production.
733
Latency caused by slow commands Redis is mostly single-threaded. So, when a request is slow to serve, all other clients must wait to be served. This waiting adds to command latencies. Redis commands also have a time complexity defined using the Big O notation.
Redis is essentially a data structure server. It supports commands and doesn't support a query language, so there is no case of using ad-hoc queries. Data access paths have to be designed, and this results in a loss of flexibility.
Because memory access is faster than disk access (0.1 μs vs. 10 ms), Redis performs extremely well when compared to disk-‐centric databases. On top of the high performance, Redis gives you two options to persist its data durably: (1) using snapshots and/or (2) using an append-‐only file.
There is no query language (only commands) and no support for a relational algebra. You cannot submit ad-hoc queries (like you can using SQL on a RDBMS). All data accesses should be anticipated by the developer, and proper data access paths must be designed. A lot of flexibility is lost.
You'll need to get some kind of back pressure for doing bulk writes from node into Redis. By default, node will queue all writes and does not enforce an upper bound on the outgoing queue size.
node_redis has a "drain" event that you can listen for to implement some rudimentary back pressure.
127
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
