Cluster Host Network¶
Overview
The cluster host network provides the physical network required for Kubernetes management and control, as well as private container networking.
Kubernetes uses logical networks for communication between containers, pods, services, and external sites. These networks are implemented over the cluster host network using the CNI service, Calico, in StarlingX.
All nodes in the cluster must be attached to the cluster host network. This network shares an interface with the management network. A container workload’s external connectivity is either through the OAM port or through other configured ports on both the controller and worker nodes, depending on containerized workload requirements.
Container network endpoints will be exposed externally with NodePort Kubernetes services. This exposes selected application containers network ports on all interfaces of both controller nodes and all worker nodes, on either the OAM interface or other configured interfaces for external connectivity on all nodes. This is typically done either directly to the application containers service or through an ingress controller service to reduce external port usage. HA would be achieved through either an external HA load balancer across two or more controller and/or worker nodes, or simply using multiple records (two or more destination controller and/or worker node IPs) for the application’s external DNS entry.
Alternatively, the cluster host network can be deployed as an external network and provides the container workload’s external connectivity as well.
Container network endpoints will be exposed externally with NodePort Kubernetes services. This exposes selected Application Containers network ports on all interfaces (e.g. external cluster host interfaces) of both controller nodes and all worker nodes. This would typically be done either directly to the Application Containers service or through an ingress controller service to reduce external port usage. HA would be achieved through either an external HA load balancer across two or more controller and/or worker nodes, or simply using multiple records (2 or more destination controller and/or worker node IPs) for the Application’s external DNS Entry.
If using an external cluster host network, container network endpoints could be exposed through BGP within the Calico CNI service. Calico BGP configuration could be modified to advertise selected Application Container services or the ingress controller service to a BGP Peer, specifying the available next hop controller and/or worker nodes’ cluster host IP Addresses.
External Connectivity from Pods¶
The preceding paragraphs cover inbound traffic, which is how external clients reach container workloads inside the cluster. This section covers the opposite direction: outbound traffic, where a pod initiates a connection to an endpoint outside the cluster, such as an external directory service, a license server, or a remote API.
A pod address is allocated from the cluster-pod network by the Calico CNI service and is not routable outside the cluster. When a pod opens a connection to an external endpoint, the traffic is source network address translated as it leaves the node, to an address belonging to the host. The external endpoint sees that host address as the source of the connection and sends its reply to it.
The host address used is the preferred source address of the route the host selects for the destination. Where the only path to the destination is the default route, this is the node’s cluster host address. Where a more specific route to the destination exists on another platform interface, the preferred source address of that interface is used instead.
Requirement. For a pod to communicate with an external endpoint, the address its traffic is sourced from must be routable from that endpoint, and that endpoint must have a working return path to it. Where the default route is used, this means the cluster host network must be configured as a globally routable network, reachable from the external endpoint.
Routing through another node is not a substitute. Configuring a node to reach the external destination through another node as a gateway does not resolve this. The gateway node forwards transit traffic without re-sourcing it, so the packet retains the originating node’s source address and the reply is still addressed to a host the external network cannot reach.
This applies to all pods. Any container workload that must reach an endpoint outside the cluster depends on this behavior, independent of which application it belongs to.
Note
The cluster-host and cluster-pod subnets are configured during bootstrap and cannot be changed after deployment. Plan for external pod connectivity before bootstrap.
If you deploy a system where neither network is routable from an external endpoint, and a workload later needs to reach that endpoint, you have only two options:
Add a separate routable platform interface on every node that may host the workload, or
Redeploy the system.
