Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

zstd block compression in MongoDB/Docker

I use docker-compose under Windows 10 like so:

version: '3'

services:
  mongo:
    image: mongo:4.2
    ports:
      - "27017:27017"
    restart: always
    volumes:
      - type: bind
        source: ${PWD}/mongod.conf
        target: /etc/mongod.conf
    entrypoint: ["mongod", "--bind_ip_all", "--config", "/etc/mongod.conf"]

My mongod.conf:

storage:
  wiredTiger:
    collectionConfig:
      blockCompressor: zstd
      configString: "allocation_size=64KB,internal_page_max=64KB,leaf_page_max=64KB"

Still I get the same disk size use when I docker exec docker_mongo_1 df both with and without zstd. With zstd:

Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sda1       65792556  49263808  13156972  79% /data/db

Without:

Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sda1       65792556  47991952  14428828  77% /data/db

(The slight variation is due to randomness of inserted data, but the number of documents and their sizes is within a few percent of each other.) I used mongodump from snappy and mongorestore into zstd to ensure most documents are untouched before running some tests.

Is a zstandard installation required on the Mongo docker image? Or am I lacking some mongodb/docker configuration? Or is it really not effective for my data? If so: why is that and can I make it effective? Or is the docker platform the problem? Or is it Docker on Windows?

PS The reason I'm not posting on a dba forum is that the question is multi-disciplinary.

like image 982
Jonas Byström Avatar asked Apr 08 '21 11:04

Jonas Byström


1 Answers

Your method of measuring disk usage by mongodb is not a good one. docker doesn't emulate a virtual machine with its own virtual disks, so df actually shows disk usage statistics of the host machine (as a result your numbers include all disk activity happening outside docker). Better run du -sh on the directory where mongodb stores its data.

like image 142
Leon Avatar answered Nov 05 '22 21:11

Leon