I was wondering whether the Linux kernel did anything special or performed any optimizations when the underlying system employs Persistent Memory Modules in Memory Mode (Near-Memory DRAM cache and NVRAM as main memory). I've tried looking in drivers/nvdimm but it seems that everything here is centered around use in App Direct mode where you MMAP in a DAX file, but in Memory Mode it's semantically and syntactically no different than using DRAM.
Does Linux employ any optimizations, or is everything handled in the hardware? Can someone link me to where any memory mode optimizations are performed in the Linux kernel? Thanks in advance!
Upstream Linux v5.2-rc1 introduced the kernel parameter page_alloc.shuffle, which is a Boolean flag that is automatically enabled if both of the following conditions are true:
page_alloc.shuffle=0 to the kernel parameter list.When this parameter is enabled, the kernel page allocator randomizes its free lists in the hope of reducing conflicts on the memory-side cache.
Examples of systems on which it's automatically enabled include KNL/KNM with MCDRAM that is partially or fully configured to run in Cache Mode and CSX/CPX with persistent memory that is partially or fully configured to run in Memory Mode. On all of these systems, there is a direct-mapped memory-side cache, although many implementation details are different.
Free list shuffling provides sustainable good performance, but not necessarily optimal or close to optimal. This is in contrast to running at high performance at first due to good memory-side cache utilization, but then the performance degrades over time due to increasing cache conflicts.
That said, I don't think anyone has tested the impact of free list shuffling on performance on a system with persistent memory running in Memory Mode, even though it's automatically enabled.
There are currently no other potential optimizations for Memory Mode accepted in the kernel.