IPv4/IPv6 Dual-Stack Network¶

Dual-stack networking facilitates the simultaneous use of both IPv4 and IPv6 addresses, or continue to use each IP version independently. To accomplish this, platform networks can be associated with 1 or 2 address pools, one for each IP version (IPv4 or IPv6). The first pool is linked to the network upon creation and cannot be subsequently removed. The second pool can be added or removed to transition the system between dual-stack and single-stack modes.

The PXE boot network is an exception, as it currently only supports IPv4. Other platform networks can be configured as either single-stack or dual-stack based on specific requirements. For internal management communication among controllers, workers, and storage nodes, the primary address pool is used as encryption is currently only available for the primary pool on the management network.

Once created, a network’s primary address pool family cannot be modified. Reinstalling the system is necessary to change it. While it is possible to edit address pool addresses for management, OAM, and admin networks, all addresses within a pool must belong to the same address family.

API and Command Line Interface Considerations¶

The following system APIs handle the association between network and address pool:

network-addrpool-assign: creates the association between a network and address pool
network-addrpool-remove: removes the association between a network and address pool
network-addrpool-list: lists all associations
network-addrpool-show: shows a specific association

The first association is done internally when the network is created using the command format:

~(keystone_admin)]$ system network-add <network_name> <network type> <dynamic> <pool_uuid>

The addrpool-modify command allows to edit all its parameters with the CLI. Follows the command format:

~(keystone_admin)]$ system addrpool-modify [--name <name>] [--network <network address>] [--prefix <network prefix>] [--ranges <ranges>] [--order <sequential | random>] [--floating-address <floating address>] [--controller0-address <controller0 address>] [--controller1-address <controller1 address>] [--gateway-address <gateway address>] <address_pool uuid>

Install a System in Dual-Stack Mode¶

It is possible to install a system in dual-stack mode by adding the secondary subnets into the bootstrap variables with comma separated values as shown in the following example:

pxeboot_subnet: 198.51.100.0/24

management_subnet: fd01::/64,198.51.0.0/24
management_start_address: fd01::2,198.51.0.2
management_end_address: fd01::ffff,198.51.0.200
management_gateway_address: fd01::1,198.51.0.1

external_oam_subnet: fd00::/64,10.20.5.0/24
external_oam_gateway_address: fd00::1,10.20.5.1
external_oam_floating_address: fd00::3,10.20.5.3
external_oam_node_0_address: fd00::4,10.20.5.4
external_oam_node_1_address: fd00::5,10.20.5.5

cluster_host_subnet: aefd:100::/64,198.51.100.0/24
cluster_pod_subnet: aefd:206::/64,172.16.0.0/16
cluster_service_subnet: aefd:207::/112,10.96.0.0/12

The order in which networks are listed determines the primary and secondary address pools. It is important to note that all primary address pools must use the same address family. For example, if the primary address pool for one subnet is IPv6, all other subnet’s primary pools must also be IPv6 (as in the example above).

Configure DNS Server¶

It is optional to configure DNS servers with both IPv4 and IPv6 addresses. This can be achieved using the command format:

~(keystone_admin)]$ system dns-modify nameservers=<IPv6 DNS server>,<IPv4 DNS server>
or
~(keystone_admin)]$ system dns-modify nameservers=<IPv4 DNS server>,<IPv6 DNS server>

If the selected DNS servers support both A and AAAA records, specifying an address for each address family is unnecessary.

Distributed Cloud Operations¶

Subclouds can be installed in a dual-stack configuration, if its version supports the feature. All operational communication between the system controller and subclouds uses the primary address pool. While the system controller and subclouds can operate in different network modes (single-stack or dual-stack), they must share the same primary address family. Geo redundancy uses the primary address pools to communicate.

Public Endpoint Considerations¶

All available public OAM endpoints can be accessed through the secondary address using the same L4 port. HA Proxy is used to map these external requests to the corresponding internal endpoints.

Modify Network Addresses¶

Supported Networks: Only the OAM, Admin, and Management networks can be modified using the addrpool-modify command during runtime.
Other Networks: For other networks, reinstallation is required to make changes.

External API and Command Line Interface Considerations¶

Deprecated

The external-OAM API is marked as deprecated but can be used to modify the OAM network primary pool.

CLI

The corresponding CLIs are:

~(keystone_admin)]$ system oam-modify <path=value> [<path=value> ...]
~(keystone_admin)]$ system oam-show

Enable Kubernetes in Dual-stack¶

To enable dual-stack functionality in Kubernetes, the OAM, cluster-host, cluster-service, and cluster-pod networks must be configured to support it. Making these changes at runtime triggers a quick restart for the kube-API-server and kube-controller-manager pods.

If converted during runtime, newly created pods will automatically receive both primary and secondary addresses. Existing pods retain their current primary addresses but will not acquire a secondary address until they are restarted. The same happens from a dual-stack to single-stack configuration, previously existing pods will retain their secondary address until restart.

Runtime Configuration¶

To add dual-stack in a running system the following sequence is suggested (in the example below the system was installed as IPv6). The network’s primary family can be seen with:

~(keystone_admin)]$ system network-list
+----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
| id | uuid...  | name            | type            | dynamic | pool_uuid                            | primary_pool_family |
+----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
| 4  | 196d...3 | multicast       | multicast       | False   | 7c445f38-067c-4b3c-a511-d8e00da5791c | IPv6                |
| 5  | 43fe...3 | cluster-host    | cluster-host    | True    | 6250edb8-15f5-4204-80f1-8e54b9e28a5a | IPv6                |
| 3  | 9996...9 | oam             | oam             | False   | b46512d7-5404-4daa-a64d-fc510e0c5864 | IPv6                |
| 6  | a374...7 | cluster-pod     | cluster-pod     | False   | f4c9560c-47e5-46bd-aff5-18642831b1da | IPv6                |
| 7  | afc1...d | cluster-service | cluster-service | False   | a6366aab-b3c1-4947-97e5-f5171e0e2f3e | IPv6                |
| 1  | b565...9 | mgmt            | mgmt            | True    | 412aebff-9a86-40b1-a379-752f00a0c3a0 | IPv6                |
| 2  | bbb1...2 | pxeboot         | pxeboot         | True    | 05fde56d-f26a-4ea4-8b32-1ebf868743e2 | IPv4                |
+----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+

Configure OAM Network¶

Add an address pool for OAM:

~(keystone_admin)]$ system addrpool-add oam-ipv4 171.168.204.0 24 --order random --ranges 171.168.204.1-171.168.204.254 --floating-address 171.168.204.1 --controller0-address 171.168.204.2 --controller1-address 171.168.204.3

Then assign the newly created pool to the OAM network:

~(keystone_admin)]$ system network-addrpool-assign oam oam-ipv4

If the system is AIO-SX, the new configuration is applied immediately, otherwise it is necessary to lock/unlock both controllers.

Configure Cluster (pod/service/host) Network¶

The cluster networks in dual-stack converts kubernetes to dual-stack operation, in this case, first make sure the OAM network is already configured in dual-stack and then start by adding the correspondent pools:

~(keystone_admin)]$ system addrpool-add cluster-pod-subnet-ipv4 172.16.0.0 16 --order random --ranges 172.16.0.1-172.16.254.254
~(keystone_admin)]$ system addrpool-add cluster-service-subnet-ipv4 10.96.0.0 12 --order random --ranges 10.96.0.1-10.96.254.254
~(keystone_admin)]$ system addrpool-add cluster-host-subnet-ipv4 192.168.204.0 24 --order random --ranges 192.168.204.1-192.168.204.254 --floating-address 192.168.204.1 --controller0-address 192.168.204.2 --controller1-address 192.168.204.3

Then associate the new pools to each network (there is no preferred order among the three networks):

~(keystone_admin)]$ system network-addrpool-assign cluster-service cluster-service-subnet-ipv4
~(keystone_admin)]$ system network-addrpool-assign cluster-pod cluster-pod-subnet-ipv4
~(keystone_admin)]$ system network-addrpool-assign cluster-host cluster-host-subnet-ipv4

After the third cluster network receives dual-stack kubernetes and calico will be reconfigured with the kube-apiserver-controller and kube-controller-manager-controller restarts. The entire operation will be performed without the need of a node lock/unlock cycle.

Configure Management Network¶

As stated, the internal communication is done through the primary pool, but it is possible to add dual-stack configuration by first adding the new pool:

~(keystone_admin)]$ system addrpool-add management-ipv4 20.20.20.0 24 --order random --ranges 20.20.20.1-20.20.20.254 --floating-address 20.20.20.1 --controller0-address 20.20.20.2 --controller1-address 20.20.20.3

Then create the association:

~(keystone_admin)]$ system network-addrpool-assign management management-ipv4

A Configuration Out-Of-Date alarm is raised for the affected nodes and a node lock/unlock cycle will clean the alarm.

Configure Admin Network¶

This network is used by subclouds to communicate with its system-controller and that is done through the primary pool. To add a dual-stack configuration start with a new pool:

~(keystone_admin)]$ system addrpool-add admin-ipv4 30.30.30.0 24 --order random --ranges 30.30.30.1-30.30.30.254 --floating-address 30.30.30.1 --controller0-address 30.30.30.2 --controller1-address 30.30.30.3

Then create the association:

~(keystone_admin)]$ system network-addrpool-assign admin admin-ipv4

This is done in runtime on the affected controllers, no lock/unlock cycle is required.

Revert to Single-stack¶

By removing the network association with the address pool the single-stack configuration operates in a similar fashion that was done to configure dual-stack. If the configuration was done at runtime, or if a node lock/unlock cycle was required to configure dual-stack, the same happens when configuring single-stack.

To remove a address pool association with a network use network-addrpool-remove, for example:

~(keystone_admin)]$ ADDR_POOL_NAME=”cluster-pod-ipv6"
~(keystone_admin)]$ DEL=$(system network-addrpool-list | awk '$6 == $ADDR_POOL_NAME { print $2 }') && system network-addrpool-remove $DEL