OpenvSwitch Operator Application

The OpenvSwitch Operator application (openvswitch) deploys and manages OVS instances on StarlingX worker nodes using a Kubernetes-native operator pattern. It provides declarative configuration of OVS bridges, ports, and per-node settings through CRDs.

The application consists of two main components:

OVS Manager Operator

A Deployment that runs on control-plane nodes. It watches for OVS custom resources (OVSBridge, OVSPort, OVSNodeConfig) and reconciles the desired state with the actual OVS configuration on each node via the OVS Agent.

OVS Agent Operator

A DaemonSet that runs on every node labeled with ovs-node=enabled. Each pod contains three containers:

• ovs-agent-operator — the controller that receives instructions from the OVS Manager and configures the local OVS instance accordingly.

• ovsdb-server — the OVS database server managing the switch configuration database.

• ovs-vswitchd — the OVS forwarding daemon that implements the configured bridges and ports.

An init container (ovs-init) bootstraps the OVS database on pod startup.

The application supports three CRD types:

OVSBridge

Defines a bridge with optional STP, MAC aging, and node selector. Targets all nodes matching a label selector.

OVSPort

Defines a port on a named bridge with VLAN configuration, STP settings, and optional BFD-enabled interfaces.

OVSNodeConfig

Combines bridge and port definitions for a specific node, allowing per-node customization of the OVS topology (generated by OVS Manager, not editable by the user).

The provided application uses kernel bridging underneath; therefore, all SR-IOV VF configurations must use netdevice drivers. The use of VFIO (DPDK) is not supported.

Install OpenvSwitch Operator Application

Before installing the application, ensure the target worker nodes are labeled with the ovs-node=enabled label. The OVS Agent DaemonSet uses this label as a node selector to determine where to schedule pods.

1. Label the target nodes:

   ~(keystone_admin)]$ system host-label-assign <hostname> ovs-node=enabled
   

2. Upload the application:

   ~(keystone_admin)]$ system application-upload openvswitch
   

3. Apply the application:

   ~(keystone_admin)]$ system application-apply openvswitch
   

Note

The ovs-node=enabled label must be assigned to at least one host before applying the application. If no hosts have this label, the OVS Agent DaemonSet will not schedule any pods.

Set up Helm Override Variables

The openvSwitch application supports Helm overrides for both the ovs-agent and ovs-manager charts. Overrides are set using the system helm-override-update command.

~(keystone_admin)]$ system helm-override-update openvswitch ovs-agent openvswitch \
   --set <key>=<value>

The following table describes the available override variables for the ovs-agent chart:

Parameter	Default	Description
ovsAgent.networks	""	Comma-separated list of network-attachment-definition (NAD) JSON objects to attach additional interfaces to the OVS pod. Each entry must have a name field referencing an existing NAD in the openvswitch namespace. Example: {"name":"nad1","interface":"port1"},{"name":"nad2","interface":"port2"}
ovsAgent.resources.requests	cpu: 10m, memory: 64Mi	Resource requests for the OVS agent container. When using SR-IOV interfaces, include the corresponding resource names (e.g., intel.com/pci_sriov_net_<nad_name>: '1').
ovsAgent.resources.limits	cpu: 500m, memory: 128Mi	Resource limits for the OVS agent container. Must include matching SR-IOV resource names when using SR-IOV interfaces.
ovsAgent.healthHostPort	19081	Host port for the health probe endpoint (accessible at localhost:<port>/readyz).
ovsAgent.statusHostPort	19082	Host port for the OVS status endpoint (accessible at localhost:<port>/status).
ovsAgent.hostAccess.enabled	false	Enables a host-access network-attachment-definition that provides the OVS pod connectivity to the host network via a Linux bridge.
ovsAgent.hostAccess.bridgeFailover.pfInterface	""	The sysinv interface name of the SR-IOV PF to use for bridge failover (e.g., sriov0). Required when hostAccess.enabled is true.
ovsContainer.ovsdbLogLevel	warn	Log level for the ovsdb-server container. Valid values: off, emer, err, warn, info, dbg.
ovsContainer.vswitchdLogLevel	warn	Log level for the ovs-vswitchd container. Valid values: off, emer, err, warn, info, dbg.

Example: Attaching SR-IOV interfaces

~(keystone_admin)]$ system helm-override-update openvswitch ovs-agent openvswitch \
   --set ovsAgent.networks='{"name":"ovs-bridge1"\,"interface":"port1"}\,{"name":"ovs-bridge2"\,"interface":"port2"}' \
   --set ovsAgent.resources.requests.intel\\.com/pci_sriov_net_ovs_bridge1='1' \
   --set ovsAgent.resources.limits.intel\\.com/pci_sriov_net_ovs_bridge1='1' \
   --set ovsAgent.resources.requests.intel\\.com/pci_sriov_net_ovs_bridge2='1' \
   --set ovsAgent.resources.limits.intel\\.com/pci_sriov_net_ovs_bridge2='1'

Alternatively, use a YAML file for complex overrides:

~(keystone_admin)]$ system helm-override-update openvswitch ovs-agent openvswitch \
   --values /path/to/overrides.yaml

Where overrides.yaml contains:

ovsAgent:
  networks: '{"name":"ovs-bridge1","interface":"port1"},
             {"name":"ovs-bridge2","interface":"port2"}'
  resources:
    requests:
      intel.com/pci_sriov_net_ovs_bridge1: '1'
      intel.com/pci_sriov_net_ovs_bridge2: '1'
    limits:
      intel.com/pci_sriov_net_ovs_bridge1: '1'
      intel.com/pci_sriov_net_ovs_bridge2: '1'

All the referenced NetworkAttachmentDefinitions need to be created previously to applying to the system. After updating overrides, re-apply the application:

~(keystone_admin)]$ system application-apply openvswitch

Uninstall OpenvSwitch Operator Application

Follow the steps below to uninstall the OpenvSwitch Operator application:

1. Remove the application:

   ~(keystone_admin)]$ system application-remove openvswitch
   

2. Delete the application:

   ~(keystone_admin)]$ system application-delete openvswitch
   

Note

During removal, the application lifecycle hook automatically cleans up OVS webhook configurations and removes finalizers from OVS custom resources to ensure that resource deletion is not blocked.

Connect Internal Traffic to the OpenvSwitch Pod

The OVS pod runs in its own network namespace and, by default, is isolated from both the host platform network and other pods. Two mechanisms are available to connect traffic into and out of the OVS bridges running inside the pod: host platform access and pod-to-pod access via bridge CNI.

Host Platform Access via OpenvSwitch

Host platform access allows the host operating system to communicate directly with the OVS bridge inside the ovs-agent pod. This is achieved through a host-access network-attachment-definition that creates a Linux bridge (ovs0) shared between the host and the pod via a veth pair.

When ovsAgent.hostAccess.enabled is set to true, the application:

• Creates a host-access NetworkAttachmentDefinition in the openvswitch namespace using a bridge CNI plugin with bridge name ovs0.

• Attaches a host0 interface to the OVS pod connected to the ovs0 Linux bridge on the host.

• Activates bridge failover: a script manages the SR-IOV PF interface membership in the Linux bridge to prevent duplicate packets when the OVS pod veth is active.

• The systemd service ovs-access-monitor starts to keep track of the ovs-agent pod health and configuration status, in case of failure it switches the ovs0 bridge membership connection from the veth device (created by the bridge-cni) to the selected pfInterface.

• The configuration status will be decided by two factors:

  • inside the virtual-switch, the host0’s ovsport needs to be attached to an ovsbridge.

  • This ovsbridge needs to have at least one extra ovsport configured, so it can send traffic to the outside interface (but it cannot identify if the selected ovsport is the correct one).

Prerequisites

• The host must have an SR-IOV PF interface configured in sysinv with class pci-sriov and type ethernet.

• An upper ethernet interface with ovs_access=true must be configured on top of the SR-IOV PF interface.

• The node must be labeled with ovs-node=enabled.

Configuration steps

1. Configure the SR-IOV PF interface and the ovs-access upper interface on each target host using the StarlingX system commands.

   ~(keystone_admin)]$ system host-if-modify controller-0 eno8603 -c pci-sriov \
      -n sriov0 -N 16 --vf-driver netdevice
   ~(keystone_admin)]$ system host-if-add controller-0 base0 ethernet sriov0 \
      -c platform --ovs-access true
   

2. Set the Helm overrides to enable host access:

   ~(keystone_admin)]$ system helm-override-update openvswitch ovs-agent openvswitch \
      --set ovsAgent.hostAccess.enabled=true \
      --set ovsAgent.hostAccess.bridgeFailover.pfInterface=<sysinv_interface_name>
   

   Replace <sysinv_interface_name> with the name of the SR-IOV PF interface as configured in sysinv (e.g., sriov0).

   Select a pci-sriov interface that will participate in a datanetwork to be used as one of the virtual-switch ports.

   There is no need to add the host access’ bridge-cni to ovsAgent.networks, this will be done automatically.

   In the example below, both the host access interface base0, and the sriov-vf interface pf1 are using sriov0 (the NetworkAttachmentDefinition pf1nad needs to be created prior to the application apply):

   ~(keystone_admin)]$ system host-if-modify controller-0 eno8603 -c pci-sriov \
      -n sriov0 -N 16 --vf-driver netdevice
   ~(keystone_admin)]$ system host-if-add controller-0 base0 ethernet sriov0 \
      -c platform --ovs-access true
   ~(keystone_admin)]$ system datanetwork-add pf1dn vlan
   ~(keystone_admin)]$ system host-if-add controller-0 pf1 vf sriov0 -c pci-sriov \
      -N 1 --vf-driver=netdevice
   ~(keystone_admin)]$ system interface-datanetwork-assign controller-0 pf1 pf1dn
   

   Add the override values to match the provided configuration (note that the host access NetworkAttachmentDefinition does not need to be added to the ovsAgent.networks field, this will be done automatically, only the external virtual-switch ports are required).

   ovsAgent:
     hostAccess:
       enabled: true
       bridgeFailover:
         pfInterface: sriov0
     networks: '{"name":"pf1nad","interface":"port1"}'
     resources:
       requests:
         intel.com/pci_sriov_net_pf1dn: '1'
       limits:
         intel.com/pci_sriov_net_pf1dn: '1'
   

   Then update the Helm override.

   ~(keystone_admin)]$ system helm-override-update openvswitch ovs-agent openvswitch \
       --values /path/to/overrides.yaml
   

3. Apply the application.

   ~(keystone_admin)]$ system application-apply openvswitch
   

4. Once the OVS pod is running, configure a port on the OVS bridge to connect to the host0 interface by creating an OVSNodeConfig or OVSPort custom resource. For example:

   apiVersion: openvswitch.starlingx.io/v1
   kind: OVSBridge
   metadata:
     name: br0
     namespace: openvswitch
   spec:
     bridgeName: br0
     rstpEnable: true
     macAgingTime: 300
     macTableSize: 2048
   ---
   apiVersion: openvswitch.starlingx.io/v1
   kind: OVSPort
   metadata:
     name: host-access-port
     namespace: openvswitch
   spec:
     name: br0-host0
     bridgeName: br0
     vlanMode: access
     vlanTag: 705
     stpEnable: false
     interfaces:
       - name: host0 # interface is available when ovsAgent.hostAccess.enabled=true
   ---
   apiVersion: openvswitch.starlingx.io/v1
   kind: OVSPort
   metadata:
     name: external-access-port
     namespace: openvswitch
   spec:
     name: br0-port1
     bridgeName: br0
     vlanMode: trunk
     vlanTag: 705
     stpEnable: false
     interfaces:
       - name: port1  # interface name comes from ovsAgent.networks
   

   Apply the resource:

   $ kubectl apply -f host-access-port.yaml
   

5. Optionally, platform networks can be added on top of the ovs-access interface.

   ~(keystone_admin)]$ system host-if-add controller-0 oam0 vlan base0 -V 705 -c platform
   ~(keystone_admin)]$ system interface-network-assign controller-0 base0 pxeboot
   ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 oam
   ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 mgmt
   

Pod Access via OpenvSwitch

Pod-to-pod connectivity through the OVS bridge is achieved by using a bridge CNI network-attachment-definition on both the application pod and the ovs-agent pod, pointing to the same Linux bridge name. This creates a shared Layer 2 segment between the two pods.

How it works

Both the OVS pod and the application pod attach to the same bridge CNI. The bridge CNI creates a Linux bridge on the host and connects a veth interface from each pod to that bridge. Traffic from the application pod traverses the Linux bridge into the OVS pod, where it can be forwarded through OVS bridge ports to external networks.

Configuration steps

1. Create a NetworkAttachmentDefinition that uses the bridge CNI plugin. If ovsAgent.hostAccess.enabled=true was selected this new bridge-cni cannot use the config field “bridge” with the “ovs0” value as this is reserved for host access, see below:

   apiVersion: "k8s.cni.cncf.io/v1"
   kind: NetworkAttachmentDefinition
   metadata:
     name: ovs-bridge-access
     namespace: openvswitch
   spec:
     # cannot use {"bridge": "ovs0"} if ovsAgent.hostAccess.enabled=true
     config: '{
               "cniVersion": "1.0.0",
               "name": "ovs-bridge-access",
               "type": "bridge",
               "bridge": "ovs-br1"
              }'
   

2. Reference this NAD in the ovsAgent.networks Helm override so that the OVS pod attaches to the bridge:

   ~(keystone_admin)]$ system helm-override-update openvswitch ovs-agent openvswitch \
            --set ovsAgent.networks='{"name":"ovs-bridge-access"\,"interface":"pod-link0"}'
   

3. In the application pod, add the same NAD annotation:

   apiVersion: v1
   kind: Pod
   metadata:
     name: my-app-pod
     annotations:
       k8s.v1.cni.cncf.io/networks: '[{"name":"ovs-bridge-access",
                                       "interface":"ovs-link0"}]'
   spec:
     containers:
       - name: app
         image: my-app-image
   

4. Apply the application to pick up the override change:

   ~(keystone_admin)]$ system application-apply openvswitch
   

Note

Pods with SR-IOV resources should prefer to connect directly to the desired interfaces rather than using the bridge CNI method. Direct SR-IOV attachment is more efficient as it bypasses the host Linux bridge and provides near-native I/O performance with lower latency.

Multi-node installations

In a multi-node deployment, the OVS Agent runs on every node that carries the ovs-node=enabled label. Bridges and ports are distributed across nodes using the OVSBridge resource’s nodeSelector field and per-node OVSPort resources that reference interfaces available on specific hosts.

Label nodes

Each node that should participate in the OVS topology must be labeled. You can use additional labels beyond ovs-node=enabled to create subgroups of nodes for different bridge topologies:

~(keystone_admin)]$ system host-label-assign worker-0 ovs-node=enabled
~(keystone_admin)]$ system host-label-assign worker-1 ovs-node=enabled
~(keystone_admin)]$ system host-label-assign worker-0 ovs-role=fronthaul
~(keystone_admin)]$ system host-label-assign worker-1 ovs-role=fronthaul

The ovs-node=enabled label is required for the DaemonSet to schedule the OVS Agent pod on a node. Additional labels (e.g., ovs-role=fronthaul) can be used in the OVSBridge resource’s nodeSelector to target a subset of OVS-enabled nodes.

Target bridges to specific nodes

The OVSBridge resource supports a nodeSelector field that controls which nodes the bridge is created on. This uses the standard Kubernetes label selector semantics (matchLabels and matchExpressions).

There are several common patterns for targeting bridges:

All OVS-enabled nodes — deploy the bridge everywhere the agent runs:

apiVersion: openvswitch.starlingx.io/v1
kind: OVSBridge
metadata:
  name: br-fronthaul
  namespace: openvswitch
spec:
  bridgeName: br-fronthaul
  rstpEnable: true
  macAgingTime: 300
  macTableSize: 2048

A subset of nodes by role — use additional custom labels to group nodes:

apiVersion: openvswitch.starlingx.io/v1
kind: OVSBridge
metadata:
  name: br-fronthaul
  namespace: openvswitch
spec:
  bridgeName: br-fronthaul
  rstpEnable: true
  macAgingTime: 300
  macTableSize: 2048
  nodeSelector:
    matchLabels:
      ovs-node: enabled
      ovs-role: fronthaul

This creates br-fronthaul only on nodes labeled with both ovs-node=enabled and ovs-role=fronthaul.

Use matchExpressions for flexible selection — target nodes using set-based operators (In, NotIn, Exists, DoesNotExist):

apiVersion: openvswitch.starlingx.io/v1
kind: OVSBridge
metadata:
  name: br-backhaul
  namespace: openvswitch
spec:
  bridgeName: br-backhaul
  rstpEnable: true
  nodeSelector:
    matchExpressions:
      - key: ovs-node
        operator: In
        values: ["enabled"]
      - key: ovs-role
        operator: In
        values: ["backhaul", "combined"]

This creates br-backhaul on any node labeled ovs-node=enabled whose ovs-role is either backhaul or combined.

Multiple bridges on different node groups — deploy distinct bridge topologies to different sets of nodes:

# Bridge for fronthaul nodes only
apiVersion: openvswitch.starlingx.io/v1
kind: OVSBridge
metadata:
  name: br-fronthaul
  namespace: openvswitch
spec:
  bridgeName: br-fronthaul
  rstpEnable: true
  nodeSelector:
    matchLabels:
      ovs-role: fronthaul
---
# Bridge for backhaul nodes only
apiVersion: openvswitch.starlingx.io/v1
kind: OVSBridge
metadata:
  name: br-backhaul
  namespace: openvswitch
spec:
  bridgeName: br-backhaul
  rstpEnable: true
  nodeSelector:
    matchLabels:
      ovs-role: backhaul

Define per-node ports

OVSPort resources reference a bridge by name and are reconciled on every node where that bridge exists. Since each node may have different physical interfaces, use distinct OVSPort resources with node-specific naming to distinguish configurations. The interface names in the OVSPort spec correspond to the Multus interface names attached to the pod via ovsAgent.networks.

The following example shows port configuration for a two-node fronthaul topology where each node has an uplink (midhaul0), a cross-link to the peer (protect0), and a downlink (fronthaul0):

# --- Ports for worker-0 ---
apiVersion: openvswitch.starlingx.io/v1
kind: OVSPort
metadata:
  name: worker0-midhaul0
  namespace: openvswitch
spec:
  name: midhaul0
  bridgeName: br-fronthaul
  vlanMode: trunk
  vlanTrunks:
    - 706
  stpEnable: true
  stpPathCost: 100
  interfaces:
    - name: midhaul0
      bfdEnable: true
      bfdMinTx: 300
      bfdMinRx: 300
      bfdMult: 3
---
apiVersion: openvswitch.starlingx.io/v1
kind: OVSPort
metadata:
  name: worker0-protect0
  namespace: openvswitch
spec:
  name: protect0
  bridgeName: br-fronthaul
  vlanMode: trunk
  vlanTrunks:
    - 706
  stpEnable: true
  stpPathCost: 200
  interfaces:
    - name: protect0
      bfdEnable: true
      bfdMinTx: 300
      bfdMinRx: 300
      bfdMult: 3
---
apiVersion: openvswitch.starlingx.io/v1
kind: OVSPort
metadata:
  name: worker0-fronthaul0
  namespace: openvswitch
spec:
  name: fronthaul0
  bridgeName: br-fronthaul
  vlanMode: access
  vlanTag: 706
  stpEnable: false
  interfaces:
    - name: fronthaul0

# --- Ports for worker-1 ---
apiVersion: openvswitch.starlingx.io/v1
kind: OVSPort
metadata:
  name: worker1-midhaul0
  namespace: openvswitch
spec:
  name: midhaul0
  bridgeName: br-fronthaul
  vlanMode: trunk
  vlanTrunks:
    - 706
  stpEnable: true
  stpPathCost: 100
  interfaces:
    - name: midhaul0
      bfdEnable: true
      bfdMinTx: 300
      bfdMinRx: 300
      bfdMult: 3
---
apiVersion: openvswitch.starlingx.io/v1
kind: OVSPort
metadata:
  name: worker1-protect0
  namespace: openvswitch
spec:
  name: protect0
  bridgeName: br-fronthaul
  vlanMode: trunk
  vlanTrunks:
    - 706
  stpEnable: true
  stpPathCost: 200
  interfaces:
    - name: protect0
      bfdEnable: true
      bfdMinTx: 300
      bfdMinRx: 300
      bfdMult: 3
---
apiVersion: openvswitch.starlingx.io/v1
kind: OVSPort
metadata:
  name: worker1-fronthaul0
  namespace: openvswitch
spec:
  name: fronthaul0
  bridgeName: br-fronthaul
  vlanMode: access
  vlanTag: 706
  stpEnable: false
  interfaces:
    - name: fronthaul0

Apply the resources:

$ kubectl apply -f br-fronthaul.yaml
$ kubectl apply -f worker0-ports.yaml
$ kubectl apply -f worker1-ports.yaml

Note

Port names in the OVSPort metadata must be unique across the cluster. Use a node-specific prefix (e.g., worker0-, worker1-) to avoid naming collisions. The spec.name field defines the actual port name inside OVS and can be the same across nodes since each node runs its own independent OVS instance.

Note

The interface names used in OVSPort specs (e.g., midhaul0, protect0, fronthaul0) must match the interface field values configured in the ovsAgent.networks Helm override. These are the Multus interface names that appear inside the OVS Agent pod.