Host Device Assignment

KubeVirt provides a mechanism for assigning host devices to a virtual machine. This mechanism is generic and allows various types of PCI devices, such as accelerators (including GPUs) or any other devices attached to a PCI bus, to be assigned. It also allows Linux Mediated devices, such as pre-configured virtual GPUs to be assigned using the same mechanism.

Preparation of Mediated Devices such as vGPU

In general, configuration of Mediated devices (mdevs), such as vGPUs, should be done according to vendor directions. KubeVirt can now facilitate the creation of the Mediated devices/vGPUs on the cluster nodes. This assumes that the required vendor driver is already installed on the nodes.

List Permitted Devices

Administrators can control which host devices are exposed and permitted to be used in the cluster. Permitted host devices in the cluster will need to be listed in KubeVirt Custom Resource (CR) by its vendor:product selector for PCI devices or Mediated device names. See the example yaml segment below, that would be added to the KubeVirt CR with kubectl edit kubevirt -n kubevirt.

configuration:
  permittedHostDevices:
    pciHostDevices:
    - pciVendorSelector: "10DE:1EB8"
      resourceName: "nvidia.com/TU104GL_Tesla_T4"
      externalResourceProvider: true
    - pciVendorSelector: "8086:6F54"
      resourceName: "intel.com/qat"
    mediatedDevices:
    - mdevNameSelector: "GRID T4-1Q"
      resourceName: "nvidia.com/GRID_T4-1Q"

Note

pciVendorSelector is a PCI vendor ID and product ID tuple in the form vendor_id:product_id. This tuple can identify specific types of devices on a host. For example, the identifier 10de:1eb8, shown above, can be found using lspci.

$ lspci -nnv|grep -i nvidia
65:00.0 3D controller [0302]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1eb8] (rev a1)

Start VM Using vGPU

Host devices can be assigned to VMs via the gpus and hostDevices fields. The deviceNames can reference both PCI and Mediated device resource names.

kind: VirtualMachineInstance
spec:
  domain:
    devices:
      gpus:
      - deviceName: nvidia.com/TU104GL_Tesla_T4
        name: gpu1
      - deviceName: nvidia.com/GRID_T4-1Q
        name: gpu2
      hostDevices:
      - deviceName: intel.com/qat
        NAME: QUICKACCESS1