Deploy a Combined Software and Kubernetes Upgrade Across Subclouds

After a software release has been deployed on the System Controller, you can orchestrate a combined software deployment and Kubernetes upgrade across the subclouds. This applies both operations to each subcloud in a single orchestration pass, eliminating the need for a separate Kubernetes upgrade orchestration afterward.

For a DCManager combined software deployment and Kubernetes upgrade across subclouds:

  • for AIO-SX subclouds, ‘combined’ upgrade will be done. For details on how the combined strategy works on standalone systems, see Orchestrated Combined Host Software Deployment and Kubernetes Upgrade.

  • for all other duplex subclouds, legacy serial software platform upgrade deployment followed by kubernetes upgrade will be done (as ‘combined’ upgrade is not supported for multi-node subcloud types).

Note

The --with-delete option has been renamed to --delete and --delete-only has been renamed to --cleanup. The old option names are deprecated and will be removed in a future release.

Prerequisites

The following conditions must be met for the orchestrated combined software deployment and Kubernetes upgrade across the subclouds to be successful.

  • All target subclouds are managed, online and healthy. The Kubernetes cluster on each subcloud is fully operational and there are no management affecting alarms.

  • The target software release has been uploaded and deployed on the System Controller.

  • The target Kubernetes version is available on the subclouds. This version is included with the target software release, but it must be available on each subcloud before the target software release is deployed.

    Note

    For upgrades to StarlingX r13, the target Kubernetes version is made available by deploying the FROM-side patch required for software upgrades to r13.

    You can confirm availability on a subcloud with system kube-version-list command.

  • Subcloud certificates are up-to-date and will not expire during the deployment.

  • Prestage the new software release and container images to subclouds before the maintenance window to reduce software deployment time. See Orchestrate Subcloud Prestage Using the CLI.

    • By default, the dcmanager sw-deploy-strategy create command assumes subclouds have been prestaged beforehand and will fail the new software deployment otherwise.

    • Specify --with-prestage option with the sw-deploy-strategy create command to include prestaging in the same strategy along with the sw-deploy operation.

  • Ensure that controller-0 is the active controller on each subcloud.

Note

The combined software deployment and Kubernetes upgrade strategy is only supported for major release upgrades.

Procedure

  1. Review software sync status of the subclouds.

    After the new software release has been deployed on the System Controller, wait for approximately 60 seconds for the software_sync_status of all subclouds to be updated.

    ~(keystone_admin)]$ dcmanager subcloud list

  2. To create a combined sw-deploy and Kubernetes upgrade strategy, use the dcmanager sw-deploy-strategy create command with the --kube-upgrade option.

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy create \
[--subcloud-apply-type <type>] \
[--max-parallel-subclouds <i>] \
[--stop-on-failure <level>] \
[--group group] \
[--with-prestage] \
[--sysadmin-password] \
--release-id RELEASE_ID \
[--kube-upgrade KUBE_VERSION] \
[--snapshot] \
[--delete] \
[--rollback] \
[--cleanup] \
[<subcloud>]

    where:

    subcloud-apply-type

    parallel or serial — determines whether the subclouds will be processed in parallel or serially.

    If this is not specified using the CLI, the values for subcloud_update_type defined for each subcloud group will be used by default.

    max-parallel-subclouds

    Sets the maximum number of subclouds that can be upgraded in parallel (default 2).

    If this is not specified using the CLI, the values for max_parallel_subclouds defined for each subcloud group will be used by default.

    stop-on-failure

    true (default) or false — determines whether orchestration failure for a subcloud prevents application to subsequent subclouds.

    group

    Optionally pass the name or ID of a subcloud group. This results in a strategy that is only applied to all subclouds in the specified group.

    with-prestage

    Enables orchestration to prestage and then deploy the new software on subclouds that lack the required software and container images.

    sysadmin-password

    Pass the sysadmin password (optional). Required when using --with-prestage.

    release-id

    Specifies the target software release to deploy.

    kube-upgrade

    Specifies the target Kubernetes version to upgrade to.

    snapshot

    Creates a snapshot of the main logical volumes so that, in the case of a rollback, the snapshots can be restored in order to speed up the overall rollback procedure.

    delete

    Specifies that the software deployment and the Kubernetes upgrade should be finalized and deleted at the end of a successful subcloud deployment. With this option, the subclouds’ deployments cannot be rolled back.

    rollback

    Specifies that sw-deploy-strategy should run a rollback of the existing (un-deleted) software deployment on the subclouds.

    This is valid after a failed sw-deploy-strategy on a subcloud or after a successful sw-deploy-strategy on a subcloud that did not delete the software deployment using –delete or –cleanup.

    cleanup

    Specifies that sw-deploy-strategy should only finalize the kubernetes upgrade and run the software deployment cleanup operation on subclouds. This command is used to clean up a previous default sw-deploy-strategy where the subclouds’ software deployments were not deleted.

    For example:

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy create --release-id RELEASE_ID --kube-upgrade KUBE_VERSION
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| strategy type          | sw-deploy                  |
| subcloud apply type    | None                       |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| release_id             | <RELEASE_ID>               |
| snapshot               | True                       |
| rollback               | False                      |
| delete_option          | delete                     |
| with_prestage          | None                       |
| kube_upgrade           | <KUBE_VERSION>             |
| state                  | initial                    |
| created_at             | 2026-07-02T21:24:39.650531 |
| updated_at             | None                       |
+------------------------+----------------------------+

  3. To show the settings for the strategy, use the dcmanager sw-deploy-strategy show command.

    For example:

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy show
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| strategy type          | sw-deploy                  |
| subcloud apply type    | None                       |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| release_id             | <RELEASE_ID>               |
| snapshot               | True                       |
| rollback               | False                      |
| delete_option          | delete                     |
| with_prestage          | None                       |
| kube_upgrade           | <KUBE_VERSION>             |
| state                  | initial                    |
| created_at             | 2026-07-02T21:24:39.650531 |
| updated_at             | None                       |
+------------------------+----------------------------+

  4. Review the strategy steps for the subclouds.

    To show the subclouds that will be processed when the strategy is applied, use the dcmanager strategy-step list command. For example:

    ~(keystone_admin)]$ dcmanager strategy-step list
+----------------------+-------+---------+---------+------------+-------------+
| cloud                | stage | state   | details | started_at | finished_at |
+----------------------+-------+---------+---------+------------+-------------+
| subcloud-1           |     1 | initial |         | None       | None        |
| subcloud-2           |     1 | initial |         | None       | None        |
+----------------------+-------+---------+---------+------------+-------------+

    Note

    All the subclouds that are included in the same stage will be deployed in parallel.

  5. To apply the strategy, use the dcmanager sw-deploy-strategy apply command.

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy apply
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| strategy type          | sw-deploy                  |
| subcloud apply type    | None                       |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| release_id             | <RELEASE_ID>               |
| snapshot               | True                       |
| rollback               | False                      |
| delete_option          | delete                     |
| with_prestage          | None                       |
| kube_upgrade           | <KUBE_VERSION>             |
| state                  | applying                   |
| created_at             | 2026-07-02T21:24:39.650531 |
| updated_at             | None                       |
+------------------------+----------------------------+

  6. To show the step currently being performed on each of the subclouds, use the dcmanager strategy-step list command.

    For example:

    ~(keystone_admin)]$ dcmanager strategy-step list
+------------------+-------+-----------------------+---------+----------------------------+----------------------------+
| cloud            | stage | state                 | details | started_at                 | finished_at                |
+------------------+-------+-----------------------+---------+----------------------------+----------------------------+
| subcloud-1       |     1 | complete              |         | 2026-07-02 21:25:10.342429 | 2026-07-02 22:10:41.054664 |
| subcloud-2       |     1 | apply VIM sw-deploy   |         | 2026-07-02 21:25:10.342429 | None                       |
+------------------+-------+-----------------------+---------+----------------------------+----------------------------+

  7. To show the step currently being performed on a specific subcloud, use the dcmanager strategy-step show command.

    ~(keystone_admin)]$ dcmanager strategy-step show <subcloud>

  8. When all the subclouds indicate they have entered either the complete or failed state, delete the strategy using the dcmanager sw-deploy-strategy delete command.

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy delete
+------------------------+----------------------------+
| Field                  | Value                      |
+------------------------+----------------------------+
| strategy type          | sw-deploy                  |
| subcloud apply type    | None                       |
| max parallel subclouds | 2                          |
| stop on failure        | False                      |
| release_id             | <RELEASE_ID>               |
| snapshot               | True                       |
| rollback               | False                      |
| delete_option          | delete                     |
| with_prestage          | None                       |
| kube_upgrade           | <KUBE_VERSION>             |
| state                  | deleting                   |
| created_at             | 2026-07-02T21:24:39.650531 |
| updated_at             | None                       |
+------------------------+----------------------------+

Postrequisites

After a successful combined software deployment and Kubernetes upgrade:

  • Validate that subclouds are online and in sync. This will only happen if the --delete option was specified.

  • If --delete was not specified during strategy creation, clean up the deployment artifacts by creating a cleanup strategy:

    ~(keystone_admin)]$ dcmanager sw-deploy-strategy create --cleanup

    Then apply and delete the cleanup strategy as described in the steps above.

    Note

    The --cleanup option cannot be combined with --release-id, --kube-upgrade, or --rollback.

Error Handling During an Orchestrated Combined Deployment

If a failure occurs, follow the general steps below:

  1. Allow the failed strategy to complete on its own.

  2. Check the output using dcmanager strategy-step list command.

  3. To see the cause of failure for a subcloud, use the dcmanager subcloud errors <subcloud-name/subcloud-id> command.

  4. Resolve the condition that caused the failure.

  5. Retry the orchestration by deleting the failed strategy and creating a new one for the failed subcloud(s).