databasecompressionquestdbzfs

Enable QuestDB compression on Mac


I want to use ZFS compression on my local environment, using a Macbook Pro M1. For my production environment I didn't have any issues by following the docs, as I have a dedicated EBS volume for the QDB_ROOT, but on my Macbook I have only my internal drive, and I don't know how I could create the ZFS unit there.

Is there any way I can do something like this on my local disk?

zpool create -m legacy -o feature@lz4_compress=enabled autoexpand=on -O compression=lz4 -t volume1 questdb-pool sdf

Solution

  • On Mac you can create a disk image using your local disk, then you can use that image. It took me a few attempts to get the right combination of parameters so I could use ZFS on top of the image, but this works and I can confirm data is compressed also on my development environment.

    Edit: On the original post I forgot to add I had to install openzfs via brew install openzfs. Thanks user Victor for pointing this out!

    # Create the disk image without a filesystem
    hdiutil create -size 400m -type SPARSEBUNDLE -layout NONE -nospotlight zfsQuestdb
    
    # Attach the disk image without mounting
    hdiutil attach -nomount zfsQuestdb.sparsebundle
    
    # Find the device name (assuming it's the last one, adjust if necessary)
    DISK=$(diskutil list | grep "/dev/disk" | tail -1 | awk '{print $1}')
    
    # Create the ZFS pool
    sudo zpool create qdb_root $DISK
    
    # Set ZFS properties
    sudo zfs set compression=lz4 qdb_root
    
    # Create the mounting directories in the home directory and assigning permissions for the local user
    mkdir -p ~/data/qdb_root
    sudo chown -R $(id -u):$(id -g) ~/data/qdb_root
    
    
    # Create ZFS filesystem and set mount point
    sudo zfs create -o mountpoint=/Users/$(whoami)/data/qdb_root qdb_root/data
    
    # Verify the setup
    zfs list
    zpool status