What is Redis change its own configurations

Question

Redis change its own config dir to /etc/cron.d and dbfile to ntp instead of default configuration. Once we restart the redis it will reset to /var/lib/redis and dump.rdb but after awhile, it gives "Failed opening the RDB file" error

Default dire and rdb file has correct permission and redis only allow for internal IPs.

cli output

127.0.0.1:6381> CONFIG GET dir
1) "dir"
2) "/etc/cron.d"
127.0.0.1:6381> CONFIG GET "dbfilename"
1) "dbfilename"
2) "ntp"

/var/log/redis/redis-server.log

3204:M 21 May 16:07:19.124 * Background saving terminated with success
3204:M 21 May 16:12:18.962 * 10000 changes in 60 seconds. Saving...
3204:M 21 May 16:12:18.967 * Background saving started by pid 25469
25469:C 21 May 16:12:20.931 * DB saved on disk
25469:C 21 May 16:12:20.934 * RDB: 3 MB of memory used by copy-on-write
3204:M 21 May 16:12:20.968 * Background saving terminated with success

3204:M 21 May 16:17:21.082 * 10 changes in 300 seconds. Saving...
3204:M 21 May 16:17:21.088 * Background saving started by pid 25865
25865:C 21 May 16:17:22.800 * DB saved on disk
25865:C 21 May 16:17:22.803 * RDB: 3 MB of memory used by copy-on-write
3204:M 21 May 16:17:22.891 * Background saving terminated with success
3204:M 21 May 16:17:43.669 # Failed opening the RDB file root (in server root dir /var/spool/cron) for saving: Read-only file system
3204:M 21 May 16:17:45.320 # Failed opening the RDB file ntp (in server root dir /etc/cron.d) for saving: Read-only file system
3204:M 21 May 16:22:23.086 * 10 changes in 300 seconds. Saving...
3204:M 21 May 16:22:23.092 * Background saving started by pid 26264
26264:C 21 May 16:22:23.093 # Failed opening the RDB file ntp (in server root dir /etc/cron.d) for saving: Read-only file system
3204:M 21 May 16:22:23.194 # Background saving error
3204:M 21 May 16:22:29.104 * 10 changes in 300 seconds. Saving...
3204:M 21 May 16:22:29.109 * Background saving started by pid 26265
26265:C 21 May 16:22:29.109 # Failed opening the RDB file ntp (in server root dir /etc/cron.d) for saving: Read-only file system
3204:M 21 May 16:22:29.209 # Background saving error
3204:M 21 May 16:22:35.016 * 10 changes in 300 seconds. Saving...

Answer 1

Is your server publicly accessibly over the internet?

The most likely explanation is that somebody is connecting to the redis and sending commands to reconfigure it remotely, trying to take control over the server.

There are bots scanning the internet 24/7 looking for exposed software and known vulnerabilities. Quick rule of thumb is that a new service coming up online will be discovered and attacked in less than 5 minutes. (Try running an unpatched Windows XP server and be amazed how short it lasts).

Consider that redis and potentially the whole server was compromised. I hope for you that there was no sensitive information in this redis or that's a data breach.

Block public access, decommission the virtual machine, setup a new one from scratch.

Related redis ticket: https://github.com/antirez/redis/issues/3594