linuxcentosazure-blob-storagemarklogicmlcp

Blobfuse with Azure on Linux - how to create tmp-path on Azure


We have used below command to mount Azure Blob as folder in CentOS Linux machine

    sudo blobfuse /mnt/azureblob/ --tmp-path=/mnt/resource/blobfusetmp  
    --config-file=/home/mladmin/fuse_connection.cfg 
    -o attr_timeout=240 -o entry_timeout=240 -o nonempty #
    -o negative_timeout=120 -o allow_other

Configuration file (fuse_connection.cfg) is as follows

accountName xxx
accountKey key
containerName container

During MLCP process, we are getting kind of below error XDMP-MERGESPACE: Not merging due to disk space limitations, need=4740MB, have=2739MB

So when we looked at it, we found that issue is with temp folder disk size.

We have moved to Azure to make sure that we do not have to worry about disk size. Now -tmp-path is mandatory, so question is how we can set -tmp-path to Azure blob ?


Solution

  • For your issue, you need to understand that what does the tmp path mean, it means when you open the files in the blob that mount on your VM, then all the files opened will store in the tmp path. So the tmp path should have enough to store the open files.

    Blobfuse stores all open file contents in the temporary path. Make sure to have enough space to accommodate all open files.

    So with my suggestion, if you do not know how much space to store the open files, you can make sure the tmp path as big as your blob. For example, the blob is 5gb, so the tmp path should also have 5gb.