Host FPGA Configuration - Intel N3000¶
Note
This guide was replaced by: N3000 FPGA Overview
This guide describes how to configure and integrate an Intel N3000 FPGA PAC on StarlingX.
Overview¶
The Intel FPGA PAC N3000 contains two Intel XL710 NICs, memory, and an Intel FPGA. The system discovers and inventories the device as a NIC, with the XL710 ports available in the host port list and host interface list.
Update device images¶
The Intel FPGA PAC N3000 as shipped from the factory is expected to have production BMC and factory images. The following procedure describes how to update the user image on a host.
Device image types:
root-key: The root-key image sets the main authentication key on the hardware.
functional-key: The functional device image performs the desired work on behalf of the application. If a rootkey device image has been written to the hardware, then the functional image will only be accepted if it has been signed by a CSK generated from the root key which has not been revoked.
key-revocation: The key-revocation device image will revoke a CSK. If a root-key device image has been written to the hardware, then the key-revocation device image will only be accepted if it has been signed by the root key.
The following items are specific to the Intel FPGA PAC N3000:
The root-key image is called the root entry hash bitstream and can only be set once.
The functional device image is known as the user image.
The key-revocation device image is known as the CSK ID cancellation bitstream.
CSKs are revoked by specifying an integer CSK ID.
128 CSK ID cancellation slots exist.
For the Intel FPGA PAC N3000, a CSK is revoked by specifying an integer ID, and all CSKs with that ID will be revoked. Writing the root-key device image or a key-revocation device image is essentially permanent. Reverting to factory status requires physical access to the card and specialized equipment.
Upload the device image.
To upload a root-key device image:
~(keystone_admin)$ system device-image-upload imagefile root-key pci_vendor pci_device --key-signature key_signature --name imagename --description description --image-version version
To upload a functional device image:
~(keystone_admin)$ system device-image-upload imagefile functional pci_vendor pci_device --functional bitstream_id --name imagename --description description -image-version version
To upload a revocation key device image:
~(keystone_admin)$ system device-image-upload imagefile key-revocation pci_vendor pci_device --revoke-key-id revoke_key_id --name imagename -- description description --image-version version
where:
imagefile # The filepath of the binary device image file. pci_vendor # The hexadecimal string identifying the PCI vendor ID of the device this image applies to. pci_device # The hexadecimal string identifying the PCI device ID of the device this image applies to. key_signature # A hexadecimal string identifying the root key device image. revoke_key_id # A decimal key ID for the key revocation device image. bitstream_id # A hexadecimal string of the functional device image. name # The name of the device image (optional). description # The description of the device image (optional). image-version # The version of the device image (optional).
Assign a device label to the device.
Labels are key-value pairs that are assigned to host PCI devices and are used to specify attributes of the devices. Labels can be used to identify certain properties of the PCI devices where the same device image can be used.
The command syntax is:
system host-device-label-assign [--overwrite] hostname_or_id pci_name_or_address name=value [name=value ...]
Overwrite the label using the
--overwrite
option. This option is not allowed while the image update is in progress after runninghost-device-image-update
. Once assigned, a device label can be referenced by multipledevice-image-apply
commands.Apply the device image on one or all supported devices.
Note
A device firmware update in progress alarm is raised once the first device image is applied.
The
system device-image-state-list
will show the status of the device as pending.Apply a device image to all supported devices:
~(keystone_admin)$ system device-image-apply image_uuid
Alternatively, apply a device image to devices with a specified label:
~(keystone_admin)$ system device-image-apply image_uuid key1=value1
Write pending device images on the host to hardware.
~(keystone)admin)$ system host-device-image-update hostname
Note
This operation currently supports one pending device image at a time.
Any previously-attempted device image writes for this host that are in a failed state will be reset to pending and retried.
Root and revocation key updates can be expected to take 1-2 minutes.
Functional image updates can take approximately 40 minutes for the Intel FPGA PAC N3000.
Once a device update is complete,
system device-image-state-list
will show the status as completed for that device/image.Once all pending device updates for the host are complete,
system host-show hostname
will again display an empty string fordevice_image_update
.
Lock and unlock the host.
(Optional) Upload, apply, and update any additional key-revocation device images or functional device images as needed.
New device images can be uploaded as needed, and already-uploaded images can be applied with new labels. Devices can also have new labels applied to them, and any device images with matching labels will be automatically applied.
Device management commands¶
This section lists the commands used to control the Intel FPGA PAC N3000.
Listing uploaded device images
system device-image-list
Listing device labels
system host-device-label-list hostname_or_id devicename_or_address
Removing device labels
system host-device-label-remove hostname_or_id key
Remove a device image
To remove a device image from all devices
system device-image-remove image_uuid
To remove the device image from all devices with a matching label
system device-image-remove image_uuid key1=value1
Initiating a Device Image Update for a Host
system host-device-image-update hostname_or_host_ID
Displaying the status of device images
system device-image-state-list
Enable Forward Error Correction¶
The Intel FPGA PAC N3000 supports FEC capabilities, which are exposed as a PCI device. The PCI device can be used by a DPDK enabled container application to perform accelerated 5G LDPC encoding and decoding operations.
After the FPGA device is programmed, the list of host devices shows the FEC device with device ID 0xd8f, as shown below.
system host-device-list <worker-node>
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
| name | address | class id | vendor id | device id | class name | vendor name | device name | numa_node | enabled |
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
| pci_0000_b7_00_0 | 0000:b7:00.0 | 120000 | 8086 | 0d8f | Processing accelerators | Intel Corporation | Device 0d8f | 1 | True |
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
...
To enable the FEC device for SR-IOV, set the number of VFs and set the appropriate userspace drivers for the PF and VF.
For example:
system host-lock <worker>
system host-device-modify <worker> <name> --driver <vf driver> --vf-driver <vf driver> -N <number of vfs>
system host-unlock <worker>
The supported PF driver(s) are:
- igb_uio
The supported VF driver(s) are currently:
- igb_uio
- vfio
To pass the FEC device to a container, enter the following requests/limits into the pod specification:
intel.com/intel_fpga_fec: '<number of vfs>'
For example:
apiVersion: v1
kind: Pod
metadata:
name: 5gnr
annotations:
k8s.v1.cni.cncf.io/networks: '[
{ "name": "sriov1" }
]'
spec:
restartPolicy: Never
containers:
- name: 5gnr
image: "5gnr-image"
volumeMounts:
- mountPath: /mnt/huge-1048576kB
name: hugepage
stdin: true
tty: true
resources:
requests:
memory: 4Gi
intel.com/intel_fpga_fec: '1'
intel.com/pci_sriov_net_datanetwork_a: '1'
limits:
hugepages-1Gi: 4Gi
memory: 4Gi
intel.com/intel_fpga_fec: '1'
intel.com/pci_sriov_net_datanetwork_a: '1'
volumes:
- name: hugepage
emptyDir:
medium: HugePages
Configure NICs for SR-IOV¶
You can configure the Intel XL710 NICs for SR-IOV by first identifying the NICs on the Intel FPGA PAC N3000 using the following command:
system host-port-list <worker>
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
| uuid | name | type | pci address | device | processor | accelerated | device type |
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
| 6c79c0d0-0463-4551-a19a-24d52a9403c6 | enp177s0f0 | ethernet | 0000:b1:00.0 | 0 | 1 | False | Device [0d58] |
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
...
Next, set the number of VFs and set the appropriate userspace drivers for the VF.
For example:
system host-lock <worker>
system host-if-list -a <worker>
system host-if-modify <worker> <interface name or uuid> -c pci-sriov --vf-driver <vf driver> -N <number of vfs>
system interface-datanetwork-assign <worker> <interface> <datanetwork>
system host-unlock <worker>
The supported VF driver(s) are currently:
- vfio
- netdevice
Note
If --vf-driver
is not specified in the system host-if-modify
command, then netdevice
(kernel driver) will be assigned.