Configure Intel Wireless FEC Accelerators using SR-IOV FEC operator

This section provides the instructions for installing the SR-IOV FEC operator that provides detailed configurability of Wireless FEC Accelerators on StarlingX (AIO-SX).

About this task

The SR-IOV FEC Operator for Intel Wireless FEC Accelerators supports the following vRAN FEC accelerators:

  • Intel® vRAN Dedicated Accelerator ACC100.

  • Intel® FPGA Programmable Acceleration Card N3000.

  • Intel® vRAN Dedicated Accelerator ACC200.

Prerequisites

  • The system has been provisioned and unlocked.

Procedure

  1. Source the platform environment.

    $ source /etc/platform/openrc
    ~(keystone_admin)$
    
  2. Upload the SR-IOV FEC Operator.

    ~(keystone_admin)$ system application-upload /usr/local/share/applications/helm/sriov-fec-operator-<version>.tgz
    +---------------+-------------------------------------+
    | Property      | Value                               |
    +---------------+-------------------------------------+
    | active        | False                               |
    | app_version   | 1.0-1                               |
    | created_at    | 2022-09-29T19:47:29.427225+00:00    |
    | manifest_file | fluxcd-manifests                    |
    | manifest_name | sriov-fec-operator-fluxcd-manifests |
    | name          | sriov-fec-operator                  |
    | progress      | None                                |
    | status        | uploading                           |
    | updated_at    | None                                |
    +---------------+-------------------------------------+
    
  3. Configure a different resource name for FEC devices as desired.

    • To change the resource name for ACC100, use the following command:

      ~(keystone_admin)$ system helm-override-update sriov-fec-operator sriov-fec-operator sriov-fec-system --set env.SRIOV_FEC_ACC100_RESOURCE_NAME=intel_acc100_fec
      
  4. Apply the SR-IOV FEC Operator.

    ~(keystone_admin)$ system application-apply sriov-fec-operator
    +---------------+-------------------------------------+
    | Property      | Value                               |
    +---------------+-------------------------------------+
    | active        | False                               |
    | app_version   | 1.0-1                               |
    | created_at    | 2022-09-29T19:47:29.427225+00:00    |
    | manifest_file | fluxcd-manifests                    |
    | manifest_name | sriov-fec-operator-fluxcd-manifests |
    | name          | sriov-fec-operator                  |
    | progress      | None                                |
    | status        | applying                            |
    | updated_at    | 2022-09-29T19:47:33.599867+00:00    |
    +---------------+-------------------------------------+
    
    ~(keystone_admin)$ system application-show sriov-fec-operator
    +---------------+-------------------------------------+
    | Property      | Value                               |
    +---------------+-------------------------------------+
    | active        | True                                |
    | app_version   | 1.0-1                               |
    | created_at    | 2022-09-29T19:47:29.427225+00:00    |
    | manifest_file | fluxcd-manifests                    |
    | manifest_name | sriov-fec-operator-fluxcd-manifests |
    | name          | sriov-fec-operator                  |
    | progress      | completed                           |
    | status        | applied                             |
    | updated_at    | 2022-09-29T19:50:27.543655+00:00    |
    +---------------+-------------------------------------+
    
  5. Verify that all the operator pods are up and running.

    $ kubectl get pods -n sriov-fec-system
    NAME                                            READY   STATUS    RESTARTS        AGE
    accelerator-discovery-svh87                     1/1     Running   0               3m26s
    sriov-device-plugin-j54hh                       1/1     Running   0               3m26s
    sriov-fec-controller-manager-77bb5b778b-bjmr8   2/2     Running   0               3m28s
    sriov-fec-daemonset-stnjh                       1/1     Running   0               3m26s
    
  6. List all the nodes in the cluster with FEC accelerators installed.

    $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system
    NAME           CONFIGURED
    controller-0   NotRequested
    
  7. Find the PCI address of the PF of SR-IOV FEC accelerator device to be configured.

    • ACC100

      $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system controller-0 -o yaml
      apiVersion: sriovfec.intel.com/v2
      kind: SriovFecNodeConfig
      metadata:
        creationTimestamp: "2022-08-25T01:33:35Z"
        generation: 1
        name: controller-0
        namespace: sriov-fec-system
        resourceVersion: "8298897"
        selfLink: /apis/sriovfec.intel.com/v2/namespaces/sriov-fec-system/sriovfecnodeconfigs/controller-0
        uid: dcab90d9-2fe2-4769-81b0-fdd54e96e287
      spec:
        physicalFunctions: []
      status:
        conditions:
        - lastTransitionTime: "2022-08-25T01:33:35Z"
            message: ""
            observedGeneration: 1
            reason: NotRequested
            status: "False"
            type: Configured
        inventory:
          sriovAccelerators:
          - deviceID: 0d5c
            driver: ""
            maxVirtualFunctions: 16
            pciAddress: "0000:17:00.0"
            vendorID: "8086"
            virtualFunctions: []
      
    • N3000

      $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system controller-0 -o yaml
      apiVersion: sriovfec.intel.com/v2
      kind: SriovFecNodeConfig
      metadata:
        creationTimestamp: "2022-10-21T18:17:55Z"
        generation: 1
        name: controller-0
        namespace: sriov-fec-system
        resourceVersion: "1996828"
        selfLink: /apis/sriovfec.intel.com/v2/namespaces/sriov-fec-system/sriovfecnodeconfigs/controller-0
        uid: 05db8606-8236-4efd-99bb-7b5ca20cd02e
      spec:
        physicalFunctions: []
      status:
        conditions:
        - lastTransitionTime: "2022-10-21T18:17:55Z"
          message: ""
          observedGeneration: 1
          reason: NotRequested
          status: "False"
          type: Configured
        inventory:
          sriovAccelerators:
          - deviceID: 0d8f
            driver: ""
            maxVirtualFunctions: 8
            pciAddress: 0000:1c:00.0
            vendorID: "8086"
            virtualFunctions: []
      
    • ACC200

      $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system controller-0 -o yaml
      apiVersion: sriovfec.intel.com/v2
      kind: SriovFecNodeConfig
      metadata:
        creationTimestamp: "2022-10-21T18:31:41Z"
        generation: 1
        name: controller-0
        namespace: sriov-fec-system
        resourceVersion: "2144487"
        selfLink: /apis/sriovfec.intel.com/v2/namespaces/sriov-fec-system/sriovfecnodeconfigs/controller-0
        uid: e4e536fc-a777-4e26-974d-71226d43c4ed
      spec:
        physicalFunctions: []
      status:
        conditions:
        - lastTransitionTime: "2022-10-21T18:31:41Z"
          message: ""
          observedGeneration: 1
          reason: NotRequested
          status: "False"
          type: Configured
        inventory:
          sriovAccelerators:
          - deviceID: 57c0
            driver: ""
            maxVirtualFunctions: 16
            pciAddress: 0000:f7:00.0
            vendorID: "8086"
            virtualFunctions: []
      
  8. Apply the FEC device configuration.

    1. ACC100 device configuration.

      • The maximum number of VFs that can be configured for ACC100 is 16 VFs.

      • There are 8 queue groups available which can be allocated to any available operation (4GUL/4GDL/5GUL/5GDL) based on the numQueueGroups parameter.

      • The product of numQueueGroups x numAqsPerGroups x aqDepthLog2 x numVfBundles must be less than 32K.

      • The following example creates 1 VF, configures ACC100’s 8 queue groups; allocating 4 queue groups for 5G Uplink and another 4 queue groups for 5G Downlink.

        apiVersion: sriovfec.intel.com/v2
        kind: SriovFecClusterConfig
        metadata:
          name: config
          namespace: sriov-fec-system
        spec:
          priority: 1
          nodeSelector:
            kubernetes.io/hostname: controller-0
          acceleratorSelector:
            pciAddress: 0000:17:00.0
          physicalFunction:
            pfDriver: "vfio-pci"
            vfDriver: "vfio-pci"
            vfAmount: 1
            bbDevConfig:
              acc100:
                # pfMode: false = VF Programming, true = PF Programming
                pfMode: false
                numVfBundles: 1
                maxQueueSize: 1024
                uplink4G:
                  numQueueGroups: 0
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink4G:
                  numQueueGroups: 0
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                uplink5G:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink5G:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
          drainSkip: true
        
      • The following example creates 2 VFs, configures ACC100’s 8 queue groups; allocating 2 queue groups each for 4G Uplink, 4G downlink, 5G Uplink and 5G downlink.

        apiVersion: sriovfec.intel.com/v2
        kind: SriovFecClusterConfig
        metadata:
          name: config
          namespace: sriov-fec-system
        spec:
          priority: 1
          nodeSelector:
            kubernetes.io/hostname: controller-0
          acceleratorSelector:
            pciAddress: 0000:17:00.0
          physicalFunction:
            pfDriver: "vfio-pci"
            vfDriver: "vfio-pci"
            vfAmount: 2
            bbDevConfig:
              acc100:
                # pfMode: false = VF Programming, true = PF Programming
                pfMode: false
                numVfBundles: 2
                maxQueueSize: 1024
                uplink4G:
                  numQueueGroups: 2
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink4G:
                  numQueueGroups: 2
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                uplink5G:
                  numQueueGroups: 2
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink5G:
                  numQueueGroups: 2
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
          drainSkip: true
        
    2. N3000 device configuration.

      • The maximum number of VFs that can be configured for N3000 is 8 VFs.

      • The maximum number of queues that can be mapped to each VF for uplink or downlink is 32.

      • The following configuration for N3000 creates 1 VF with 32 queues each for 5G uplink and 5G downlink.

        apiVersion: sriovfec.intel.com/v2
        kind: SriovFecClusterConfig
        metadata:
          name: config
          namespace: sriov-fec-system
        spec:
          priority: 1
          nodeSelector:
            kubernetes.io/hostname: controller-0
          acceleratorSelector:
            pciAddress: 0000:1c:00.0
          physicalFunction:
            pfDriver: pci-pf-stub
            vfDriver: vfio-pci
            vfAmount: 1
            bbDevConfig:
              n3000:
                # Network Type: either "FPGA_5GNR" or "FPGA_LTE"
                networkType: "FPGA_5GNR"
                # Pf mode: false = VF Programming, true = PF Programming
                pfMode: false
                flrTimeout: 610
                downlink:
                  bandwidth: 3
                  loadBalance: 128
                  queues:
                    vf0: 32
                    vf1: 0
                    vf2: 0
                    vf3: 0
                    vf4: 0
                    vf5: 0
                    vf6: 0
                    vf7: 0
                uplink:
                  bandwidth: 3
                  loadBalance: 128
                  queues:
                    vf0: 32
                    vf1: 0
                    vf2: 0
                    vf3: 0
                    vf4: 0
                    vf5: 0
                    vf6: 0
                    vf7: 0
          drainSkip: true
        
      • The following configuration for N3000 creates 2 VFs with 16 queues each, mapping 32 queues with 2 VFs for 5G uplink and another 32 queues with 2 VFs for 5G downlink.

        apiVersion: sriovfec.intel.com/v2
        kind: SriovFecClusterConfig
        metadata:
          name: config
          namespace: sriov-fec-system
        spec:
          priority: 1
          nodeSelector:
            kubernetes.io/hostname: controller-0
          acceleratorSelector:
            pciAddress: 0000:1c:00.0
          physicalFunction:
            pfDriver: pci-pf-stub
            vfDriver: vfio-pci
            vfAmount: 2
            bbDevConfig:
              n3000:
                # Network Type: either "FPGA_5GNR" or "FPGA_LTE"
                networkType: "FPGA_5GNR"
                # Pf mode: false = VF Programming, true = PF Programming
                pfMode: false
                flrTimeout: 610
                downlink:
                  bandwidth: 3
                  loadBalance: 128
                  queues:
                    vf0: 16
                    vf1: 16
                    vf2: 0
                    vf3: 0
                    vf4: 0
                    vf5: 0
                    vf6: 0
                    vf7: 0
                uplink:
                  bandwidth: 3
                  loadBalance: 128
                  queues:
                    vf0: 16
                    vf1: 16
                    vf2: 0
                    vf3: 0
                    vf4: 0
                    vf5: 0
                    vf6: 0
                    vf7: 0
          drainSkip: true
        
    3. ACC200 device configuration.

      • The maximum number of VFs that can be configured for ACC200 is 16 VFs.

      • There are 16 queue groups available which can be allocated to any available operation (4GUL/4GDL/5GUL/5GDL) based on the numQueueGroups parameter.

      • The product of numQueueGroups x numAqsPerGroups x aqDepthLog2 x numVfBundles must be less than 64K.

      • The following configuration creates 1 VF, configures ACC200’s 12 queue groups; allocating 16 queues per VF for 5G processing engine functions(5GUL/5GDL/FFT).

        apiVersion: sriovfec.intel.com/v2
        kind: SriovFecClusterConfig
        metadata:
          name: config
          namespace: sriov-fec-system
        spec:
          priority: 1
          nodeSelector:
            kubernetes.io/hostname: controller-0
          acceleratorSelector:
            pciAddress: 0000:f7:00.0
          physicalFunction:
            pfDriver: vfio-pci
            vfDriver: vfio-pci
            vfAmount: 1
            bbDevConfig:
              acc200:
                # Pf mode: false = VF Programming, true = PF Programming
                pfMode: false
                numVfBundles: 1
                maxQueueSize: 1024
                uplink4G:
                  numQueueGroups: 0
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink4G:
                  numQueueGroups: 0
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                uplink5G:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink5G:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                qfft:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
          drainSkip: true
        
      • The following configuration creates 2 VF, configures ACC200’s 16 queue groups; allocating 16 queues per VF for 4G and 5G processing engine functions(4GUL/4GDL/5GUL/5GDL/FFT).

        apiVersion: sriovfec.intel.com/v2
        kind: SriovFecClusterConfig
        metadata:
          name: config
          namespace: sriov-fec-system
        spec:
          priority: 1
          nodeSelector:
            kubernetes.io/hostname: controller-0
          acceleratorSelector:
            pciAddress: 0000:f7:00.0
          physicalFunction:
            pfDriver: vfio-pci
            vfDriver: vfio-pci
            vfAmount: 2
            bbDevConfig:
              acc200:
                # Pf mode: false = VF Programming, true = PF Programming
                pfMode: false
                numVfBundles: 2
                maxQueueSize: 1024
                uplink4G:
                  numQueueGroups: 2
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink4G:
                  numQueueGroups: 2
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                uplink5G:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                downlink5G:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
                qfft:
                  numQueueGroups: 4
                  numAqsPerGroups: 16
                  aqDepthLog2: 4
          drainSkip: true
        
    4. The SriovFecClusterConfig must be provided with spec.drainSkip: True to avoid node draining.

    5. Create and apply a SriovFecClusterConfig custom resource using the above examples as templates, setting the parameters nodeSelector:kubernetes.io/hostname and acceleratorSelector:pciAddress to select the desired device and configuring vfAmount and numVfBundles as desired.

      $ kubectl apply -f <sriov-fec-config-file-name>.yaml
      sriovfecclusterconfig.sriovfec.intel.com/config created
      

      Note

      The vfAmount and numVfBundles in SriovFecClusterConfig must be always equal for ACC100/ACC200.

  9. Verify that the FEC configuration is applied.

    • An example of ACC100 status after applying 1 VF configuration.

      $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system controller-0 -o yaml
      apiVersion: sriovfec.intel.com/v2
      kind: SriovFecNodeConfig
      metadata:
        creationTimestamp: "2022-09-29T19:49:59Z"
        generation: 2
        name: controller-0
        namespace: sriov-fec-system
        resourceVersion: "2935834"
        selfLink: /apis/sriovfec.intel.com/v2/namespaces/sriov-fec-system/sriovfecnodeconfigs/controller-0
        uid: 1a39b2a6-7512-4f44-8a64-083df7e480f3
      spec:
        physicalFunctions:
        - bbDevConfig:
            acc100:
              downlink4G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 0
              downlink5G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 4
              maxQueueSize: 1024
              numVfBundles: 1
              pfMode: false
              uplink4G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 0
              uplink5G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 4
          pciAddress: "0000:17:00.0"
          pfDriver: vfio-pci
          vfAmount: 1
          vfDriver: vfio-pci
      status:
        conditions:
        - lastTransitionTime: "2022-09-29T20:33:13Z"
          message: Configured successfully
          observedGeneration: 2
          reason: Succeeded
          status: "True"
          type: Configured
        inventory:
          sriovAccelerators:
          - deviceID: 0d5c
            driver: vfio-pci
            maxVirtualFunctions: 16
            pciAddress: "0000:17:00.0"
            vendorID: "8086"
            virtualFunctions:
            - deviceID: 0d5d
              driver: vfio-pci
              pciAddress: "0000:18:00.0"
      
    • An example of N3000 status after applying 2 VFs configuration.

      $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system controller-0 -o yaml
      apiVersion: sriovfec.intel.com/v2
      kind: SriovFecNodeConfig
      metadata:
        creationTimestamp: "2022-10-21T18:17:55Z"
        generation: 2
        name: controller-0
        namespace: sriov-fec-system
        resourceVersion: "2011601"
        selfLink: /apis/sriovfec.intel.com/v2/namespaces/sriov-fec-system/sriovfecnodeconfigs/controller-0
        uid: 05db8606-8236-4efd-99bb-7b5ca20cd02e
      spec:
        drainSkip: true
        physicalFunctions:
        - bbDevConfig:
            n3000:
              downlink:
                bandwidth: 3
                loadBalance: 128
                queues:
                  vf0: 16
                  vf1: 16
                  vf2: 0
                  vf3: 0
                  vf4: 0
                  vf5: 0
                  vf6: 0
                  vf7: 0
              flrTimeout: 610
              networkType: FPGA_5GNR
              uplink:
                bandwidth: 3
                loadBalance: 128
                queues:
                  vf0: 16
                  vf1: 16
                  vf2: 0
                  vf3: 0
                  vf4: 0
                  vf5: 0
                  vf6: 0
                  vf7: 0
          pciAddress: 0000:1c:00.0
          pfDriver: pci-pf-stub
          vfAmount: 2
          vfDriver: vfio-pci
      status:
        conditions:
        - lastTransitionTime: "2022-10-21T19:35:18Z"
          message: Configured successfully
          observedGeneration: 2
          reason: Succeeded
          status: "True"
          type: Configured
        inventory:
          sriovAccelerators:
          - deviceID: 0d8f
            driver: pci-pf-stub
            maxVirtualFunctions: 8
            pciAddress: 0000:1c:00.0
            vendorID: "8086"
            virtualFunctions:
            - deviceID: 0d90
              driver: vfio-pci
              pciAddress: 0000:1c:00.1
            - deviceID: 0d90
              driver: vfio-pci
              pciAddress: 0000:1c:00.2
      
    • An example of ACC200 status after applying 1 VF configuration.

      $ kubectl get sriovfecnodeconfigs.sriovfec.intel.com -n sriov-fec-system controller-0 -o yaml
      apiVersion: sriovfec.intel.com/v2
      kind: SriovFecNodeConfig
      metadata:
        creationTimestamp: "2022-10-21T18:31:41Z"
        generation: 3
        name: controller-0
        namespace: sriov-fec-system
        resourceVersion: "2159562"
        selfLink: /apis/sriovfec.intel.com/v2/namespaces/sriov-fec-system/sriovfecnodeconfigs/controller-0
        uid: e4e536fc-a777-4e26-974d-71226d43c4ed
      spec:
        drainSkip: true
        physicalFunctions:
        - bbDevConfig:
            acc200:
              downlink4G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 2
              downlink5G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 4
              maxQueueSize: 1024
              numVfBundles: 1
              qfft:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 4
              uplink4G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 2
              uplink5G:
                aqDepthLog2: 4
                numAqsPerGroups: 16
                numQueueGroups: 4
          pciAddress: 0000:f7:00.0
          pfDriver: vfio-pci
          vfAmount: 1
          vfDriver: vfio-pci
      status:
        conditions:
        - lastTransitionTime: "2022-10-21T19:48:26Z"
          message: Configured successfully
          observedGeneration: 3
          reason: Succeeded
          status: "True"
          type: Configured
        inventory:
          sriovAccelerators:
          - deviceID: 57c0
            driver: vfio-pci
            maxVirtualFunctions: 16
            pciAddress: 0000:f7:00.0
            vendorID: "8086"
            virtualFunctions:
            - deviceID: 57c1
              driver: vfio-pci
              pciAddress: 0000:f7:00.1
      
  10. Modify FEC Cluster config.

    1. To further modify FEC device configuration, make desired modifications to the sriov-fec custom resource file and re-apply.

      $ kubectl apply -f <sriov-fec-config-file-name>.yaml
      sriovfecclusterconfig.sriovfec.intel.com/config configured
      
  11. Delete SriovFecClusterConfig.

    $ kubectl delete -f <sriov-fec-config-file-name>.yaml
    sriovfecclusterconfig.sriovfec.intel.com "config" deleted
    
  12. Configure VFIO for PF interface.

    SR-IOV FEC operator also supports vfio-pci driver for PF interface.

    If the vfio-pci driver is used to bind the PF interface, then a UUID token must be configured as a VFIO_TOKEN to both PF and VF interfaces.

    • For the PF interface, the VFIO_TOKEN is configured by SR-IOV FEC operator and has the default value of 02bddbbf-bbb0-4d79-886b-91bad3fbb510

    • It is highly recommended to change the default vfio-token when configuring the accelerator in vfio mode (ie., vfio-pci driver for PF interface).

    • The VFIO_TOKEN could be changed by setting SRIOV_FEC_VFIO_TOKEN before application Apply with system helm-override-update.

    • This example sets the SRIOV_FEC_VFIO_TOKEN using uuidgen.

      ~(keystone_admin)$ system helm-override-update sriov-fec-operator sriov-fec-operator sriov-fec-system --set env.SRIOV_FEC_VFIO_TOKEN=`uuidgen`
      
    • For the VF interface, the same VFIO_TOKEN must be configured by the application. You can get the token using the command system helm-override-show sriov-fec-operator sriov-fec-operator sriov-fec-system.

    • To configure ACC100, N3000 and ACC200 in vfio mode, you should provide sriovFecClusterConfig with spec.physicalFunction.pfDriver: vfio-pci.

  13. Switch between Static method configuration and Operator method.

    1. Delete SriovFecClusterConfig.

    2. Remove sriov-fec-operator using the command system application-remove.

    3. Apply the configuration using system host-device-modify, see Enable ACC100/ACC200 Hardware Accelerators for Hosted vRAN Containerized Workloads.

Postrequisites

  • See Set Up Pods to Use SRIOV to Access ACC100/ACC200 HW Accelerators.

  • Resource Request: The resource name for FEC VFs configured with SR-IOV FEC operator must be intel.com/intel_fec_acc100 for ACC100, intel.com/intel_fec_5g for N3000 and intel.com/intel_fec_acc200 for ACC200 when requested in a pod spec unless the resource name was modified using the system helm-override-update command.

    • Resource request for ACC100.

      resources:
        requests:
          intel.com/intel_fec_acc100: '16'
        limits:
          intel.com/intel_fec_acc100: '16'
      
    • Resource request for N3000.

      resources:
        requests:
          intel.com/intel_fec_5g: '2'
        limits:
          intel.com/intel_fec_5g: '2'
      
    • Resource request for ACC200.

      resources:
        requests:
          intel.com/intel_fec_acc200: '16'
        limits:
          intel.com/intel_fec_acc200: '16'
      
  • vfio-token: (in case of vfio mode)

    An application pod can get the VFIO token through a pod environment variable.

    For example, reference the pod spec section for vfio token injection.

    env:
    - name: SRIOV_FEC_VFIO_TOKEN
      value: "02bddbbf-bbb0-4d79-886b-91bad3fbb510"
    

    Note

    The application can get the existing vfio-token using the command below, if the user updates the custom vfio-token. system helm-override-show sriov-fec-operator sriov-fec-operator sriov-fec-system

    If the vfio-token is available by default, it will not be displayed in the output file.

    Note

    Use the default vfio-token for testing purposes only.

  • Run the following command once the application pod is ready to get the PCI address of the allocated FEC device along with the VFIO token when applicable.

    • ACC100

      sysadmin@controller-0:~$ kubectl exec -ti app-pod -- env | grep FEC
      PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100=0000:32:00.0
      SRIOV_FEC_VFIO_TOKEN=02bddbbf-bbb0-4d79-886b-91bad3fbb510
      
    • ACC200

      sysadmin@controller-0:~$ kubectl exec -ti app-pod -- env | grep FEC
      PCIDEVICE_INTEL_COM_INTEL_FEC_ACC200=0000:f7:00.0
      SRIOV_FEC_VFIO_TOKEN=02bddbbf-bbb0-4d79-886b-91bad3fbb510
      
    • N3000

      sysadmin@controller-0:~$ kubectl exec -ti app-pod -- env | grep FEC
      PCIDEVICE_INTEL_COM_INTEL_FEC_5G=0000:1f:00.0
      
  • Applications that are using FEC VFs when the PF interface is bound with the vfio-pci driver, should provide the vfio-token to the VF interface.

    For example, a sample DPDK application can provide vfio-vf-token via Environment Abstraction Layer (EAL) parameters. ./test-bbdev.py -e="--vfio-vf-token=$SRIOV_FEC_VFIO_TOKEN -a$PCIDEVICE_INTEL_COM_INTEL_FEC_ACC200"