Foglight Administrators DB Performance Monitoring APM vWorkspace Virtualization
Brief about SR-IOV
SR-IOV is a specification that can present single PCIe device (this is called Physical Function) as multiple independent PCIe devices (each one is called Virtual Function) to OS or hypervisor. Dedicating a virtual function to a VM can be accomplished by a mechanism called PCI pass-through with which I/O operations can be done by accessing the virtual function directly from the VM.
By providing a virtual function directly to a VM, the hypervisor’s virtual switch (vSwitch) is no longer required to process network traffic. This hypervisor bypass increases network throughput, lowers latency, and reduces overall CPU utilization.
Limitations in ESXi 5.1 and 5.5 releases
In ESXi 5.1/5.5 host, we have limited number of interrupt vectors available for pass-through devices. So the ESXi host supports limited number of virtual functions. The total numbers of interrupt vectors allocated to pass-through devices are less than 128.
SR-IOV enhancements in ESXi 6.0 release
In ESXi 6.0 host, the total number of interrupt vectors can scale up to 4096 if the system is having 32 CPUs. When the host boots, devices on the host such as storage controllers, physical network adapters, and USB controllers consume a subset of the 4096 vectors. If these devices require more than 1024 vectors, the maximum number of potentially supported virtual function is reduced.
Simple formula to estimate the maximum number of virtual functions for use if all 3072 interrupt vectors is available for pass-through:
3X + 2Y < 3072
3 interrupt vectors per virtual function for Intel adapters (X) and 2 interrupt vectors per VF for Emulex adapters (Y).
The number 3072 might be smaller if other types of devices on the host use more than 1024 interrupt vectors from the total of 4096 vectors on the host.
SR-IOV in VMware ESXi
SR-IOV feature enhancements in VMware vSphere ESXi 5.5
Comparison between NPAR and SR-IOV in VMware ESXi