QAT Device Plugin Configuration¶

Intel® QuickAssist Technology (Intel® QAT) accelerates cryptographic workloads by offloading the data to hardware that is capable of optimizing those functions.

This section describes how to enable and consume the Intel QAT device plugin in StarlingX.

Prerequisites

The host should have Intel QAT hardware. Supported QAT devices are 4940, 4942, and 4946. After StarlingX is installed, do the following verification to ensure QAT devices are configured.

Verify QAT SR-IOV physical functions are configured.

$ for i in 4940 4942 4946; do lspci -d 8086:$i; done

Verify the QAT driver vfio_pci is installed.

$ lsmod | grep vfio_pci

vfio_pci               12288  0
vfio_pci_core          90112  1 vfio_pci
vfio_iommu_type1       45056  0
vfio                   61440  4 vfio_pci_core,vfio_iommu_type1,vfio_pci
irqbypass              16384  2 vfio_pci_core,kvm

Node Feature Discovery application must be installed, using the following commands.

~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/node-feature-discovery-<version>.tgz
~(keystone_admin)]$ system application-apply node-feature-discovery

Replace <version> with the latest version number.

Enable Intel QAT Device Plugin¶

The following steps should be performed to enable the Intel QAT device plugin for discovering and advertising QAT VF (Virtual Functions) resources to Kubernetes host.

Locate the application tarball in the /usr/local/share/applications/helm directory. For example:

/usr/local/share/applications/helm/intel-device-plugins-operator-<version>.tgz
Upload the application using the following command.
```
~(keystone_admin)]$ system application-upload intel-device-plugins-operator-<version>.tgz
```
Replace <version> with the latest version number.
Verify that the application has been uploaded successfully.
```
~(keystone_admin)]$ system application-list
```

Check the Helm chart status.

~(keystone_admin)]$ system helm-override-list intel-device-plugins-operator --long

Enable QAT helm chart.

~(keystone_admin)]$ system helm-chart-attribute-modify --enabled true intel-device-plugins-operator intel-device-plugins-qat intel-device-plugins-operator

Apply the application.

~(keystone_admin)]$ system application-apply intel-device-plugins-operator

Monitor the status of the application.

~(keystone_admin)]$ watch -n 5 system application-list

~(keystone_admin)]$ watch kubectl get pods -n intel-device-plugins-operator

Check the pods.

$ kubectl get pods -n intel-device-plugins-operator

NAME                                          READY STATUS  RESTARTS AGE

intel-qat-plugin-qatdeviceplugin-sample-g8n45 1/1   Running 0        34s
intel-plugin-operator-5fb6cd4864-zpjz4        1/1   Running 0        64s

Verify QAT devices by checking the node’s resource allocations. In this example, the available QAT resource is qat.intel.com/sym-dc, and both the QAT 4940 and QAT 4942 devices provide 16 virtual functions each. If both devices are present, the following command displays a total of 32 virtual functions:
```
$ kubectl describe node <node name> | grep qat.intel.com

Capacity:
---
qat.intel.com/sym-dc: 32
---
Allocatable:
---
qat.intel.com/sym-dc: 32
---
```
Note

By default, the available QAT resource is qat.intel.com/sym-dc. Other resources such as qat.intel.com/cy (both sym and asym) or qat.intel.com/asym-dc can be enabled via application overrides.

Use Intel QAT Device Plugin¶

This section describes the steps for using QAT device plugin.

Deploy a pod using the following sample pod specification file. For resource requests and limits, use the QAT resource name returned by the kubectl describe node <node_name> | grep qat.intel.com command and adjust the amounts as needed.

The qat.intel.com/<configured services>: <number of devices> field is used to configure the requested QAT virtual functions.

For a DPDK-based workload, you may need to add a hugepage request and limit.

qat-dpdk.yaml

kind: Pod
apiVersion: v1
metadata:
  name: dpdk-test-crypto-perf
spec:
  containers:
  - name: crypto-perf
    image: intel/crypto-perf:devel
    imagePullPolicy: IfNotPresent
    command: [ "/bin/bash", "-c", "--" ]
    args: [ "while true; do sleep 300000; done;" ]
    volumeMounts:
    - mountPath: /dev/hugepages
      name: hugepage
    - mountPath: /var/run/dpdk
      name: dpdk-runtime
    resources:
      requests:
        cpu: "3"
        memory: "128Mi"
        qat.intel.com/sym-dc: '4'
        hugepages-2Mi: "128Mi"
      limits:
        cpu: "3"
        memory: "128Mi"
        qat.intel.com/sym-dc: '4'
        hugepages-2Mi: "128Mi"
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
      capabilities:
        add:
          ["IPC_LOCK"]
  restartPolicy: Never
  volumes:
  - name: dpdk-runtime
    emptyDir:
      medium: Memory
  - name: hugepage
    emptyDir:
      medium: HugePages

Apply the pod specification file to create dpdk-test-crypto-perf pod.

$ kubectl apply -f qat-dpdk.yaml

Verify the pod status and the allocated QAT virtual functions.

$ kubectl get pods

NAME                  READY STATUS  RESTARTS AGE
dpdk-test-crypto-perf 1/1   Running 0        27m

$ kubectl describe pod dpdk-test-crypto-perf

Requests:
---
qat.intel.com/sym-dc: 4
---

$ kubectl describe node <controller-name>

Allocated resources:
---
qat.intel.com/sym-dc: 4
---

For more information, see: Demos and Testing.

Override Default Configuration¶

The configuration for QAT plugin defines the services that are available as resources in the kubernetes cluster. The default QAT configuration enables qat.intel.com/sym-dc as resource and is defined as follows:

qat.conf: |
  ServicesEnabled=sym;dc

The user can change the configuration to enable either sym;asym or asym;dc services via application overrides.

For example, the following example creates a config override file to enable sym;asym services:

$ cat << 'EOF' > qat-override.yml
overrideConfig:
qat.conf: |
    ServicesEnabled=sym;asym
EOF

Then, apply the override file:

$ system helm-override-update intel-device-plugins-operator intel-device-plugins-qat intel-device-plugins-operator --values qat-override.yaml

Apply intel-device-plugins-operator application:

$ system application-apply intel-device-plugins-operator

Then check the resources available. When defining sym;asym as services the resource qat.intel.com/cy becomes available:

$ kubectl describe node <node name> | grep qat.intel.com

Capacity:
---
qat.intel.com/cy: 16
---
Allocatable:
---
qat.intel.com/cy: 16
---