What influences the amount of data QuestDB writes out when it is doing "CHECKPOINT CREATE"? I noticed when I run before initiating a snapshot of the disk it uses seemingly many gigabytes.
I wonder if something is misconfigured on our side so it is forced to do more that it normally should.
When you run CHECKPOINT CREATE; the database makes a sync() system call to flush all dirty pages from the OS page cache to disk. This is done to make sure all the data and metadata is consistent at the point of taking the snapshot. Depending on your memory and database activity, this alone could explain a few gigabytes of data being written.
The statement also takes a snapshot of all the transactions for all tables at that moment in time, but this is simply metadata and it takes just a few KBs.
If you are ingesting data before you execute CHECKPOINT RELEASE, more metadata will be created to keep track of versions and offsets of files. If your ingestion is append-only, the size of those files will be small, but if you are doing out of order ingestion, or updating existing rows, the amount of data used might be significant.
Once CHECKPOINT RELEASE is called, the database will be back into its regular mode and the extra disk space will be eventually reclaimed.