Is there a way to perform a Redis GET command with the built-in Apache beam Redis I/O transform?

Question

My use case for Google Cloud Dataflow is to use Redis as a cache during the pipeline, since the transformation to occur depends on some cached data. This would mean performing Redis GET commands. The documentation for the official, built-in Redis I/O transform mentions supporting a few methods:

read - "provides a source which returns a bounded PCollection containing key/value pairs as KV"

readAll - "can be used to request Redis server using input PCollection elements as key pattern (as String)"

It looks like the readAll does not correspond to a GET command though because the input PCollection would be used to filter the result of scanning a whole Redis source, so this isn't what I'm looking for.

I was wondering if there is something I'm missing when looking at the built-in I/O transform that would enable my use case, or whether there are alternatives like open source 3rd party I/O transforms that support it. Or, is this something that is fundamentally incompatible with Apache Beam?

Answer 1

You can use RedisConnectionConfiguration. It will give you a serializable connection that you can use in your transforms.