Time Sensitive Networking

Storyboard: https://storyboard.openstack.org/#!/story/2005516

Time Sensitive Networking (TSN) is a set of evolving standards developed by IEEE to allow for time-sensitive traffic on Ethernet based LANs, which provides bounded worst-case latency. It is a key technology that targets different segments such as Automotive, Industrial etc.

TSN enabling on StarlingX will be a large effort since TSN is composed of multiple IEEE standards and varied use cases. This story will implement the initial items (based on the most commonly used standards) to integrate TSN on StarlingX to support real-time applications. The StarlingX TSN will initially support the use cases which will be further elaborated in the Use Cases section below.

Problem Description

Embedded sectors such as Automotive, Industrial, professional audio/video networking as well as blockchain and high frequency trading have emphasized the need for real-time networks.

Common LAN models are based on Internet Protocols and the IEEE 802 architecture and most of operation is best-effort. However, this is not suitable for use cases that require high / known / deterministic availability in the communication networks:

  • like circuit switching networks

  • or Fieldbuses for control networks

    • operational network (OT) != information network (IT)

    • e.g.: CAN*, EtherCAT*, Profibus*, Profinet*, …

      • lack of interoperability

To address time determinism challenges through the networks, TSN was coined by the IEEE 802.1 Working Group to cover a set of vendor-neutral and IEEE standards with the aim of guaranteeing determinism in delivering time-sensitive traffic with low and bounded latency within an infinitesimal packet loss over the network, while allowing non time-sensitive traffic to be carried through the same network.

This spec focuses on enabling the most commonly used standards on StarlingX to support real-time application.

  1. IEEE 802.1AS or time synchronization (gPTP) which is based on IEEE 1588 protocol

  2. IEEE 802.1Qav or Credit-Based Shaper (CBS)

  3. IEEE 802.1Qbv or Time Aware Shaper (TAS)

Use Case I. IEEE 1588 and IEEE 802.1AS Time Synchronization

Users require precise time synchronization with a time server across the VMs. The purpose of this synchronization is that all VMs have the same time so they can coordinate portions of time in a control loop for different purposes.

Use Case II. IEEE 802.1Qav Credit Based Shaper

Users need to ensure bounded transmission latency for time sensitive, loss-sensitive real-time data stream in some critical user scenarios. For instance, when time sensitive traffic and best effort traffic are transmitted together, users require the bandwidth and latency of time-sensitive traffic is protected in the midst of overloaded traffic on the same network, i.e. ensure time-sensitive traffic to have constant transmission rate and latency.

Use Case III. IEEE 802.1Qbv Time Aware Shaper

Users need to create a protected transmission window for scheduled traffic, which requires low and bounded transmission latency. Scheduled traffic is the term used in IEEE 802.1Qbv to refer to periodic traffic such as industrial automation control frames. This type of traffic is short in frame length and requires immediate transmission when its schedule starts.

Proposed Change

For all use cases, this spec targets to leverage PCI passthrough of the Intel® Ethernet Controller I210 NICs to the guest VMs (which is expected to be supported by Nova in the Stein release, the Nova support was done by the OpenStack community under a re-factoring/cleanup PCI activity). No special work item except user guides required in StarlingX.

This spec focuses on enable TSN on VM based workload, container support will be addressed in future spec.

More specifically,

  • Use Case I will support Precision Time Protocol (PTP) inside the VM. The implementation would be similar to the PTP bare metal host support in StarlingX: https://storyboard.openstack.org/#!/story/2002935.

  • Use Case II will use the Credit Based Shaper (CBS) (IEEE 802.1Qav) and the LaunchTime feature of I210 NIC to ensure bounded transmission latency for time sensitive, loss-sensitive real-time data stream.

  • Use Case III will use the Linux networking technology called taprio qdisc as a software implementation of Time Aware Shaper transmission gates, defined in IEEE 802.1Qbv, together with the LauchTime feature of I210 NIC to further reduce transmission latency for scheduled traffic.

Alternatives

The above use cases also be enabled through:

  1. SRIOV if hardware support available in the future

Data model impact

N/A

REST API impact

N/A

Security impact

No security impact is expected.

Other end user impact

End users will need to create instances with the specified TSN PCI passthrough devices that matches the alias from the flavor.

Performance Impact

No performance impact is expected.

Other deployer impact

None - this does not impact existing deployments.

Developer impact

No developer impact is expected.

Upgrade impact

No upgrade impact is expected.

Implementation

Assignee(s)

Primary assignee:

  • Huifeng Le

Repos Impacted

  • stx-docs

Work Items

  • Sample Nova configurations: /etc/nova/nova.conf

  • Verification of the current PCI passthrough device management in StarlingX

  • User guide on how to create VM image with TSN support

  • User guide for the introduced use cases

Dependencies

None

Testing

All use cases need to be tested after cut-over to Stein, using the image that contains the kernel support for AS/1588, Qav and Qbv.

Documentation Impact

A user doc should be introduced to help users easily setup time-sensitive applications on top of StarlingX

References

[Feature] Support for Precision Time Protocol https://storyboard.openstack.org/#!/story/2002935

Attaching physical PCI devices to guests https://docs.openstack.org/nova/latest/admin/pci-passthrough.html

History

Revisions

Release Name

Description

2019.03

Introduced