This blog is originally written by Teixeira Fabiano

 

Howdy,

Have you heard about NVDIMM-N? NVDIMM-N is a very nice feature available on Dell EMC 14G platform (click here for more details). NVDIMM-N (also known as Persistent Memory [PMEM]) is a Storage Class Memory (SCM), technology that combines flash storage and DRAM on the same memory module, enabling extraordinary performance improvement over other storage technologies such as SAS/SATA drives, SSD and NVMe. NVDIMM-N modules (288-Pin DDR4 2666MHz) are connected to a standard memory slot, taking full advantage of high bandwidth and low latency of memory bus.

NVDIMM-N can operate in two different modes on Windows Server 2016 RTM and Windows Server version 1709:

DAX: Direct Access
Block: Regular block device

DAX pretty much bypass the whole storage stack, delivering very low latency access to the application (Application must be DAX-Aware – SQL is a great example). NVDIMM still can deliver a low latency access in Block mode, however the IO still need to go through the whole storage stack.

Dell EMC presented a great demo @MS Ignite 2017 where we could see the power of NVDIMM - a SQL Server running on 14G server with DAX mode enable, delivering a fast low latency configuration also NVDIMMs configured as Cache device on Storage Spaces Direct (S2D), which also delivered a great low latency configuration. Hope you had a chance to stop by to check the demo.

MS released new Windows Server 1709 October 17th, 2017 and they added a very cool feature to Hyper-V (Compute): Storage-Class support for VMs and Virtualized Persistent Memory (vPEM). How about having this low latency technology inside a VM (Windows and Linux – not all the OSes support NVDIMM), fully utilizing your new 14G server? It would be great, right?

So, how about configuring that? Let’s do it!
 
------------------------------
Lab Configuration
------------------------------
Server: R740xd
• 2 x CPUs (Intel(R) Xeon(R) Gold 6126T CPU @ 2.60GHz)
• 2 x NVDIMM-N (16GB DDR4 2666MHz) + 12 x regular RDIMMS (also 16GB DDR4 2666MHz)
• BOSS (Boot Optimization Storage Solution) – 2xM.2 SATA SSD in HW RAID used for the OS Installation
• OS: Windows Server, version 1709

----------------------------------------------------------------------------
Storage-Class Memory support for VMs - Limitations
----------------------------------------------------------------------------
• No Migrations
• No runtime resizing
• No Thin-Provisioning or Snapshots
• Incompatible with old Hypervisors (2016 RTM or below)
• Implemented through PowerShell

---------------------------------------------------------------------
Configuring NVDIMM/PMEM in a Windows VM
---------------------------------------------------------------------
1. Enable NVDIMM in the R740xd BIOS. You can enable node interleaving on Windows Server 1709, something not possible on Windows 2016 RTM. Instead of seeing multiple SCM devices in Windows, you will see only one device if Node Interleaving is configured.
2. Install Windows Server version 1709.
3. Install the Hyper-V role.
4. Verify if SCM disk has been detected by the Hypervisor.
 


 
Follow the procedure below if OS cannot identify the SCM disk:
A) Verify Pnp Device Status (Storage Class Memory Bus).

 

B) Run the following PowerShell cmdlet to find out the PnP Device ID, if Device Status is equal to “Error”:

 

C) Create the following registry keys:

Path: HKLM\System\CurrentControlSet\Enum\ACPI\ACPI0012\<PnP Device ID>\Device Parameters\
Name: ScmBus

Path: HKLM\System\CurrentControlSet\Enum\ACPI\ACPI0012\<PnP Device ID>\Device Parameters\ScmBus\
Name: NfitValidateAllowUnalignedSPAs, Type: DWORD, Value: 1

 


D) Reboot the server.
E) Check PnP Device again in order to make sure Storage Class Memory Bus device is working properly.

 

F) Verify if SCM disk has been detected

 

5. Initialize SCM disk.



6. Create New Volume, then format it. Use the parameter -DAX $True in order to properly enable SCM for Hyper-V utilization. You won’t be able to present NVDIMM to VMs if -DAX option is not present.
 


7. Confirm that DAX is enabled.
 


8. Create a new Gen2 Virtual Machine. Install Windows 2016 RTM or Windows Server, version 1709.
9. Shutdown the VM.
 


10. Add PMEM Controller to the VM.
 


 
11. Create .vhdpmem file (new file extension). You will need to specify the -Fixed parameter. The vhdpmem disk won’t work with dynamic VHD configuration.


 
12. Attach VHDPMEM to the VM,
 
  


 

13. Start the VM
 

14. Connect to the VM (You can do everything from PS!) in order to check the SCM disk is there.
 

15. Initialize and format the SCM disk. For the SQL guys (SQL 2016 or above), if you want to take advantage of NVDIMMs, format the volume as DAX inside the VM (use the -IsDAX $True parameter).
 

----------------------------------------------------------------
Configuring NVDIMM/PMEM in a Linux VM
----------------------------------------------------------------
1. Create a VM Gen2 and install the OS (I’m running RHEL 7.4 in my lab).
2. Repeats Steps 9-13.
3. After starting the VM, connect to the VM via Console, use Bash in Windows 10 (needs to be installed 1st) or use your preferred SSH tool in order to verify if the “pmem” device has been detected.
 

For more information on how to configure Persistent Memory on Linux RHEL, click here.

Enjoy!

Sincerely,
Fabiano Teixeira

Have any comments, questions or suggestions? Please contact us on WinServerBlogs@dell.com