Aerospike namespace configuration options

Question

I've recently been getting involved in implementing an Aerospike data store into our product. We've been trying to work out the best configuration for our namespace. The requirement to persist data means we need to have a storage-engine as a device. we have specified data-in-memory as true.

My question is: does data-in-memory attempt to load ALL the backing store data into memory as the vague description implies?

Keep a copy of all data in memory always.

Or will it pay attention to the memory-size setting on the namespace and only load memory-size amount of data from the backing store?

Description of the setting was retrieved from documentation.

I have been talking to the guy who first implemented aerospike to try and find out if he knew and wasn't sure so I'm looking for clarification.

For reference my namespace config looks something like this, with an obviously smaller memory quota than backing store

namespace Test {
    replication-factor 2
    memory-size 4G
    default-ttl 0
    storage-engine device {
        file /opt/aerospike/data/Test.dat
        filesize 16G
        data-in-memory true
    }
}

Answer 1

It will keep all the data in memory. Aerospike does not yet have a partial cache implementation to keep the most used data in the provided memory.

Answer 2

Your data will only exist in memory, while the disk is used for persistence for recovery in the event of server restart. The reason the filesize is larger than the memory-size is that disk space is needed for maintenance operations such as defragmentation of blocks. Disk devices are block devices, and in the default 1MB write-block-size you can fit multiple records so operations such as defrag occur by moving records from blocks that are less than defrag-lwm-pct full. This takes extra blocks, so you need that spare capacity.