Setting up VDI machines with virtual Nvidia graphics card

From Notes_Wiki

Home > VMWare platform > VMWare Horizon > Setting up VDI machines with virtual Nvidia graphics card

In case of VDI the physical server can have graphics card such as Nvidia A40 and then we can associate smaller virtual slices of this Graphics card with VMs to give small amount of graphics memory + CPU to the VM. This can help with 3D applications performance in the VDI environment.

Downloading Nvidia software from site

To download Nvidia software from site use:

  1. Enable SSH on ESXi host
  2. SSH to ESXi host and run 
    lspci | grep -i nvidia
    Example output 
    0000:3b:00.0 3D controller: NVIDIA Corporation GA102GL [A40]
  3. As per https://www.nvidia.com/en-us/drivers/vgpu-software-driver/ go to https://www.nvidia.com/en-us/data-center/resources/vgpu-evaluation/ to get 90 day evaulation license and software download links
  4. The trial email may take 24-48 hours to arrive
  5. Go to https://nvid.nvidia.com and login with partner ID
  6. Go to Entitlements and you should see the required entitlements
  7. Go to Softwarae Downloads and download
    1. VMWare vSphere 7.0 - Product 14.1 - NVIDIA vGPU for vSphere 7.0
    2. 2022.02 64-bit license manager for windows


Setup Nvidia license server

  1. Create a VM to work as license server.
    1. Copy Java 8 RE on that VM and install
      • Set System level Environment Variable
        JAVA_HOME
        C:\Program Files\Java\jre1.8.0_333 (or other appropriate path as per enviroment)
  2. Copy "2022.02 64-bit license manager for windows" on VM and install
  3. Enable only port 7070 via firewall
    • Do not select 8080 port used for license management in firewall exception list
  4. The server is accessible at http://localhost:8080/licserver
  5. Go to https://nvid.nvidia.com and login with partner ID
    1. Go to license servers -> Create server
    2. Enable create legacy server option
    3. Get the MAC ID of license server from http://localhost:8080/licserver -> Configuration page -> Server Host ID (The one with (Ethernet))
    4. Ignore failover server related details
    5. Choose the license quantity
    6. Create server
  6. Again on nvidia portal go to License Servers -> List servers
    1. Click on the server created now
    2. Go to Actions menu and download the license file
  7. Go to http://localhost:8080/licserver on the license server.
    1. Go to License Management
    2. Upload the downlaoded bin file
    3. After successful license activation go to "Licensed Feature usage", to validate the count.


Install Driver in ESXi host for the GPU

  1. Extract "NVIDIA-GRID-vSphere-7.0-510.73.06-510.73.08-512.78.zip" downloaded for "VMWare vSphere 7.0 - Product 14.1 - NVIDIA vGPU for vSphere 7.0"
  2. Among extracted file there is vGPU file such as NVD-VGPU_510.73.06-1OEM.702.0.0.17630552_19796074.zip, extract that as

well

  1. Finally we should get a vib file such as NVIDIA_bootbank_NVIDIA-VMware_ESXi_7.0.2_Driver_510.73.06-1OEM.702.0.0.17630552.vib
  2. Copy VIB to ESXi host
  3. Put the host in maintenance mode
  4. Install VIB using: 
    cd /vmfs/volumes/vsan:52baf28e60d3aa90-4fd7fd59a4538bf7/ISO   #Change path appropriately based on environment
    

    esxcli software vib install -v $PWD/NVIDIA_bootbank_NVIDIA-VMware_ESXi_7.0.2_Driver_510.73.06-1OEM.702.0.0.17630552.vib
    

    nvidia-smi
  5. See whether reboot is required or not. If not, remove host from maintenance mode and perform the same on all other ESXi hosts


Install Required drivers in the Windows VM

  1. Go to *all* ESXi hosts in vCenter -> Configure
    1. Go to Graphics
    2. Select the graphics card and click edit
    3. Change Type to "Shared Direct"
  2. Create VM with virtual graphics
    2. Edit settings -> Add PCI device
    3. Select Nvidia Grid vGPU and select appropriate GPU profile
  3. Login into VM and install the drivers downloaded from https://nvid.nvidia.com/ - "VMWare vSphere 7.0 - Product 14.1 - NVIDIA vGPU for vSphere 7.0". The zip file has drivers for Linux and Windows VMs also along with ESXi vib.
  4. After that reboot machine. After reboot right click on Desktop gives Nvidia control panel option.
  5. In Nvidia control panel give license server IP/FQDN and port 7070
  6. After license is acquired go to task manager -> Performance -> GPU. We can see GPU with 1 GB memory.


Adding VM to Horizon

In Horizon Nvidia VGPU based machines can only be added if 3D renderer is set to "Nvidia GPU" instead of default Disabled value


Enable vMotion for vGPU based machines

By default machines with vGPU will not live migrate to other hosts even if the same graphics card with same vGPU profile is available on the target host. To enable that use:

  1. Login into vCenter
  2. Select vCenter on the left "Hosts and clusters" page. Go to Configure -> Advanced Settings
  3. Change value of 'vgpu.hotmigrate.enabled' from default false to true
  4. The change is applied immediately. After this change vMotion should work.

Refer:


Home > VMWare platform > VMWare Horizon > Setting up VDI machines with virtual Nvidia graphics card

Retrieved from "http://www.sbarjatiya.com/notes_wiki/index.php?title=Setting_up_VDI_machines_with_virtual_Nvidia_graphics_card&oldid=6611"