QAT Device Plugin Configuration¶

Intel® QuickAssist Technology (Intel® QAT) accelerates cryptographic workloads by offloading the data to hardware that is capable of optimizing those functions.

This section describes how to enable and consume the Intel QAT device plugin in StarlingX.

Prerequisites

The host should have Intel QAT hardware. Supported QAT devices are 4940 and 4942. After StarlingX is installed, do the following verification to ensure QAT devices are configured.

Verify QAT SR-IOV physical functions are configured.

$ for i in 4942 4940; do lspci -d 8086:$i; done

Verify QAT SR-IOV virtual functions are configured.

$ for i in 4943 4941; do lspci -d 8086:$i; done

$ sudo /etc/init.d/qat_service status # Must list all the virtual functions

Checking status of all devices.

There is 34 QAT acceleration device(s) in the system:

qat_dev0 - type: 4xxx, inst_id: 0, node_id: 0, bsf: 0000:f3:00.0,
#accel: 1 #engines: 9 state: up

qat_dev1 - type: 4xxx, inst_id: 1, node_id: 0, bsf: 0000:f7:00.0,
#accel: 1 #engines: 9 state: up

qat_dev2 - type: 4xxxvf, inst_id: 0, node_id: 0, bsf: 0000:f3:00.1,
#accel: 1 #engines: 1 state: up

qat_dev3 - type: 4xxxvf, inst_id: 1, node_id: 0, bsf: 0000:f3:00.2,
#accel: 1 #engines: 1 state: up

Verify the QAT driver vfio_pci is installed.

$ lsmod | grep vfio_pci

vfio_pci 69632 0
vfio_virqfd 16384 1 vfio_pci
fio 45056 4 intel_qat,vfio_mdev,vfio_iommu_type1,vfio_pci
irqbypass 16384 3 intel_qat,vfio_pci,kvm

Node Feature Discovery application must be installed, using the following commands.

~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/node-feature-discovery-<version>.tgz
~(keystone_admin)]$ system application-apply node-feature-discovery

Replace <version> with the latest version number.

Enable Intel QAT Device Plugin¶

The following steps should be performed to enable the Intel QAT device plugin for discovering and advertising QAT VF (Virtual Functions) resources to Kubernetes host.

Locate the application tarball in the /usr/local/share/applications/helm directory. For example:

/usr/local/share/applications/helm/intel-device-plugins-operator-<version>.tgz
Upload the application using the following command.
```
~(keystone_admin)]$ system application-upload intel-device-plugins-operator-<version>.tgz
```
Replace <version> with the latest version number.
Verify that the application has been uploaded successfully.
```
~(keystone_admin)]$ system application-list
```

Check the Helm chart status.

~(keystone_admin)]$ system helm-override-list intel-device-plugins-operator --long

Enable QAT helm chart.

~(keystone_admin)]$ system helm-chart-attribute-modify --enabled true intel-device-plugins-operator intel-device-plugins-qat intel-device-plugins-operator

Apply the application.

~(keystone_admin)]$ system application-apply intel-device-plugins-operator

Monitor the status of the application.

~(keystone_admin)]$ watch -n 5 system application-list

~(keystone_admin)]$ watch kubectl get pods -n intel-device-plugins-operator

Check the pods.

$ kubectl get pods -n intel-device-plugins-operator

NAME                                          READY STATUS  RESTARTS AGE

intel-qat-plugin-qatdeviceplugin-sample-g8n45 1/1   Running 0        34s
inteldeviceplugins-controller-manager-74f4c   2/2   Running 0        64s

Verify QAT devices by checking the node’s resource allocations. The QAT 4940 device and the QAT 4942 device each have 16 virtual functions. If both devices are present, the following command will display a total of 32 virtual functions:
```
$ kubectl describe node <node name> | grep qat.intel.com/sym-dc

Capacity:
---
qat.intel.com/sym-dc: 32
---
Allocatable:
---
qat.intel.com/sym-dc: 32
---
```

Use Intel QAT Device Plugin¶

This section describes the steps for using QAT device plugin.

Deploy a pod using the following sample POD specification file. The pod specification file can be modified for required resource request and limit.

The qat.intel.com/sym-dc: <number of devices> field is used to configure the requested QAT virtual functions.

For a DPDK-based workload, you may need to add a hugepage request and limit.

qat-dpdk.yaml

kind: Pod
apiVersion: v1
metadata:
  name: dpdk-test-crypto-perf
spec:
  containers:
  - name: crypto-perf
    image: intel/crypto-perf:devel
    imagePullPolicy: IfNotPresent
    command: [ "/bin/bash", "-c", "--" ]
    args: [ "while true; do sleep 300000; done;" ]
    volumeMounts:
    - mountPath: /dev/hugepages
      name: hugepage
    - mountPath: /var/run/dpdk
      name: dpdk-runtime
    resources:
      requests:
        cpu: "3"
        memory: "128Mi"
        qat.intel.com/sym-dc: '4'
        hugepages-2Mi: "128Mi"
      limits:
        cpu: "3"
        memory: "128Mi"
        qat.intel.com/sym-dc: '4'
        hugepages-2Mi: "128Mi"
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
      capabilities:
        add:
          ["IPC_LOCK"]
  restartPolicy: Never
  volumes:
  - name: dpdk-runtime
    emptyDir:
      medium: Memory
  - name: hugepage
    emptyDir:
      medium: HugePages

Apply the pod specification file to create dpdk-test-crypto-perf pod.

$ kubectl apply -f qat-dpdk.yaml

Verify the pod status and the allocated QAT virtual functions.

$ kubectl get pods

NAME                  READY STATUS  RESTARTS AGE
dpdk-test-crypto-perf 1/1   Running 0        27m

$ kubectl describe pod dpdk-test-crypto-perf

Requests:
---
qat.intel.com/sym-dc: 4
---

$ kubectl describe node <controller-name>

Allocated resources:
---
qat.intel.com/sym-dc: 4
---

For more information, see: Demos and Testing.