I have been trying to optimise the small file performance of my GlusterFS storage cluster.
A number of forum threads and blog posts seem to suggest setting the cluster.readdir-optimize property on the volume, like:
$ gluster volume get test-share cluster.readdir-optimize on
The default for this option (as of GlusterFS v3.10) seems to be off, which makes me think there must be some trade-off to having this feature enabled. However, I have not been able to find anywhere any documentation explaining exactly what this option does.
I would like to understand the function of this option before I enable it in production.
As noted in the relevant GlusterFS git repository commit message, the readdir-optimize option supports the following:
Bring in option which is supported by posix xlator to filter out directory's entries from being returned. DHT would now request non-first subvols to filter out directory entries.
I don't fully understand how this directly improves performance in GlusterFS with respect to small files. But according to the GlusterFS documentation the BD xalator performs the function of wrapping the GlusterFS block back-end and enables GlusterFS volumes to be composed of bricks which are themselves underlying logical volumes.