memoryintelcpu-architecturesolid-state-drivepersistent-memory

Confused about Intel Optane DC SSD usage as extra RAM with IMDT?


I'm a little confused about Intel Optane DC. I want that my Optane DC will be able to perform as DRAM and storage both. On the one hand, I understood that only "Intel Optane DC Persistent Memory DIMM" is able to perform as DRAM.That it because he has 2 modes (Memory mode and App-Direct Mode). On the other hand, in this link: https://www.intel.com/content/www/us/en/products/docs/memory-storage/solid-state-drives/optane-ssd-dc-p4800x-mdt-brief.html I read that "Together, DRAM and Intel® Optane™ SSDs with Intel® Memory Drive Technology emulate a single volatile memory pool". I'm confused, is Intel Optane DC SSD is able to perform as DRAM or only Intel persistent Memory DIMM?


Solution

  • Yes you can use a P4800x with Intel's IMDT (Intel Memory Drive Technology) software to give the illusion of more RAM by using the Optane DC SSD as swap space. This is what you want. IMDT sets up a hypervisor that gives the OS the illusion of DRAM + SSD as physical memory, instead of just letting the OS use it as swap space normally.

    Apparently this works well when you already have enough physical RAM for most of your working set, and IMDT has smart prefetching algorithms that try to page in ahead of when a page will be needed.

    One advantage to running the OS under the IMDT hypervisor instead of just using the SSD as swap space is that it will get the OS to use some of that extra space for pagecache (aka disk caching), instead of needing special code to use (some of) an SSD as cache for a slower disk.


    But no, it's not Optane DC Persistent Memory, that's something else.

    See also a SuperUser answer for more about Optane vs. Optane DC PM. And Hadi Brais added some nice sections to it about IMDT for Optane SSDs.

    P4800x is connected over PCI-express (as you can see in pictures on https://www.anandtech.com/show/11930/intel-optane-ssd-dc-p4800x-750gb-handson-review) for example. So it's not an NV-DIMM; you can't stick it in a DIMM socket and have the CPU access it over the memory bus. The form-factor isn't DIMM.

    As far as hardware, there are 3 things with the Optane brand name:

    See articles like https://www.techspot.com/news/79483-intel-announces-optane-dc-persistent-memory-dimms.html which put Optane DC Persistent Memory above Optane DC in the classic pyramid storage hierarchy.

    AFAIK, Optane DC PM devices only exist in a DIMM form-factor, not PCIe (and uses something like DDR4 signalling). This requires special support from the CPU because modern CPUs integrate the memory controller.

    In theory you could have a PCIe device that exposed some persistent storage in a PCIe memory region. Those are part of physical address space and can be configured as write-back cacheable. (Or can they? Mapping MMIO region write-back does not work) So they could be memory-mapped into userland virtual address space. But I don't think any PCIe Optane DC Persistent Memory devices exist, probably because PCIe command latency is (much) higher than over the DDR4 bus. Bandwidth is also lower. So it makes sense to use it as fast swap space (copying in a whole page), not as write-back cacheable physical memory where you could have cache misses waiting a very long time.

    (Margaret Bloom also comments re: block size of writes maybe being a problem.)

    i.e. you don't want a "hot" part of your working set on memory that the CPU accesses over the PCIe bus. You probably don't even want that for Optane DC PM.


    Optane / 3D XPoint is always persistent storage; it' up to software whether you take advantage of that or just use it as slower volatile RAM.

    It's not literally DRAM, that has a specific technical meaning (dynamic = data stored in tiny capacitors that need refreshing frequently). 3D XPoint isn't dynamic, and isn't even volatile. But you can use it as equivalent because 3D XPoint memory has very good write endurance (it doesn't wear out like NAND flash). If people talk about using Optane as more DRAM, they're using the term to just mean volatile RAM, filling the same role that DRAM traditionally fills.