Manual Metapackage Host Software Deployment

StarlingX software releases are composed of one or more metapackage releases. Each metapackage groups a set of packages related to a specific platform competence (e.g., infrastructure, distributed cloud, Kubernetes). Some patch releases support independent deployment of these metapackages. This topic describes how to deploy, roll back, and remove individual metapackages, or a subset of metapackages, rather than a full product release.

This topic provides the low-level software commands for deploying individual metapackages across all hosts of the local cloud.

Alternatively, you can use sw-manager sw-deploy-strategy-create/apply/delete to orchestrate the low-level software commands across all hosts of the local cloud. See Orchestrated Deployment Host Software Deployment.

In a DC environment, for deploying software to subclouds, you can use dcmanager sw-deploy-strategy-create/apply/delete on the System Controller to orchestrate the running of sw-manager sw-deploy-strategy-create/apply/delete across all subclouds managed by the System Controller. See Deploy Software Releases using the CLI.

Note

  • To abort and roll back a metapackage software deployment, follow the same general procedure as described in Manual Rollback Host Software Deployment. Note that the rollback applies to the entire deployment; all selected metapackages are rolled back together. Individual metapackage rollback is not supported.

  • To remove a metapackage deployment and return to a previously deployed full product release, follow the same general procedure as described in Manual Removal Host Software Deployment.

Prerequisites

  • A recent full backup is recommended before performing major changes to the system.

  • No active alarms are present on the system. Upgrades will not proceed if alarms are present.

  • If you are using a private container image registry for installs/updates/upgrades:

    • The private registry must be populated with any new container images required for the new software release.

    • The list of container images required by the new software release can be found in your software distribution site.

Note

For upgrades from StarlingX Release r11 to StarlingX Release r13 only:

  • The sysadmin and admin passwords must be set to the same value before starting the upgrade.

  • Before upgrading on multinode systems (AIO-DX, AIO-DX with worker node, and Standard), resize the Docker filesystem on all controllers to at least 50GB to prevent filesystem threshold alarms during the upgrade:

    ~(keystone_admin)]$ system host-fs-modify <controller-name> docker=50

  • The FEC operator stays in a CrashLoopBack state during reboot, unlock, and restore. Before the upgrade, remove the metrics_timer_config_wa_10212024.sh script to prevent this:

    ~(keystone_admin)]$ metrics_timer_config_wa_10212024.sh -r

Pre-Upgrade Deploy

Note

The pre-upgrade-deploy feature is intended for future functionality and will be fully documented in the next release, where it will be used for upgrading to that release.

Metapackage Deployment

You can select individual metapackages for deployment rather than deploying a full product release.

In the example output tables:

  • <current-release> represents the StarlingX active release version.

  • nn.nn represents the StarlingX future release version.

Procedure

  1. For a duplex (dual controller) system, switch the activity from controller-1 such that controller-0 becomes active.

    Note

    This step is not required for an AIO-SX system.

    ~(keystone_admin)]$ system host-swact controller-1

    Wait for the activity to switch to controller-0. This may take up to a minute depending on the hardware.

    Reconnect to the system using the floating OAM IP address, as the active controller has changed.

  2. Transfer the new software release files to the active controller-0.

    • For a major release, it includes the major release install ISO, software signature file, and license file.

    • For a patch release, it includes the patch release .patch archive file.

  3. Upload the new software release into the system.

    1. For a patch release:

      ~(keystone_admin)]$ software upload <filename>.patch
<release-id> is now uploaded

    2. Ensure that the new software release was successfully uploaded.

      ~(keystone_admin)]$ software list

      The output lists all software releases on the system. Confirm that the current release is in the deployed state and that the newly uploaded release is in the available state.

    3. To list the available metapackages and their states, use:

      ~(keystone_admin)]$ software metapackage list --all

+---------------------------------+-------+-----------+
| Release                         | RR    |   State   |
+---------------------------------+-------+-----------+
| base_<current-release>          | True  | deployed  |
| distcloud_<current-release>     | True  | deployed  |
| infra_<current-release>         | True  | deployed  |
| k8s-common_<current-release>    | True  | deployed  |
| k8s-v1.32.2_<current-release>   | True  | deployed  |
| k8s-v1.33.0_<current-release>   | True  | deployed  |
| k8s-v1.34.1_<current-release>   | True  | deployed  |
| k8s-v1.35.2_<current-release>   | True  | deployed  |
| swmgmt_<current-release>        | True  | deployed  |
| infra_nn.nn                     | True  | deployed  |
| base_nn.nn                      | True  | available |
| distcloud_nn.nn                 | True  | available |
| infra_nn.nn                     | True  | available |
| k8s-common_nn.nn                | True  | available |
| k8s-v1.32.2_nn.nn               | True  | available |
| k8s-v1.33.0_nn.nn               | True  | available |
| k8s-v1.34.1_nn.nn               | True  | available |
| k8s-v1.35.2_nn.nn               | True  | available |
| swmgmt_nn.nn                    | True  | available |
+---------------------------------+-------+-----------+

      Where: nn.nn represents the target future release version.

      Without the --all flag, only metapackages from the highest deployed release are shown:

      ~(keystone_admin)]$ software metapackage list

+--------------------------------+-------+----------+
| Release                        | RR    |  State   |
+--------------------------------+-------+----------+
| base_<current-release>         | True  | deployed |
| distcloud_<current-release>    | True  | deployed |
| k8s-common_<current-release>   | True  | deployed |
| k8s-v1.32.2_<current-release>  | True  | deployed |
| k8s-v1.33.0_<current-release>  | True  | deployed |
| k8s-v1.34.1_<current-release>  | True  | deployed |
| k8s-v1.35.2_<current-release>  | True  | deployed |
| swmgmt_<current-release>       | True  | deployed |
| infra_<current-release>        | True  | deployed |
+--------------------------------+-------+----------+

  4. (Optional) Select metapackages for deployment.

    This step allows you to select which metapackages to deploy before running prechecks or starting the deployment. You can select one or more metapackages.

    If you skip this step, pass the release IDs directly to the software deploy precheck and software deploy start commands instead.

    • To select individual metapackages:

      ~(keystone_admin)]$ software deploy select <metapackage-release-1> <metapackage-release-2>
Selected <metapackage-release-1> metapackage for deployment
Selected <metapackage-release-2> metapackage for deployment

    • To unselect previously selected metapackages:

      ~(keystone_admin)]$ software deploy unselect StarlingX-nn.nn
Unselected metapackage infra_nn.nn for deployment
Unselected metapackage k8s-v1.34.1_nn.nn for deployment
Unselected metapackage k8s-v1.35.2_nn.nn for deployment
Unselected metapackage swmgmt_nn.nn for deployment
Unselected metapackage distcloud_nn.nn for deployment
Unselected metapackage base_nn.nn for deployment
Unselected metapackage k8s-v1.33.0_nn.nn for deployment
Unselected metapackage k8s-v1.32.2_nn.nn for deployment
Unselected metapackage k8s-common_nn.nn for deployment

      You can also unselect all selected metapackages at once:

      ~(keystone_admin)]$ software deploy unselect --all

    Note

    Selection validation rules:

    • All selected metapackages must belong to the same product release version.

    • Certain metapackages can only be deployed as part of a full product release selection and cannot be selected individually.

    • The current running release cannot be selected for deployment.

    • Removing a previous release is only supported by selecting the full product release (individual metapackage removal is not supported).

  5. Run software deployment prechecks and confirm that the system is healthy.

    If no metapackages are specified in the software deploy precheck command, the precheck runs against the previously selected metapackages (selected in the previous step). If metapackages are specified in the software deploy precheck command, the metapackage selection is reset to those specified metapackages and the precheck runs against them.

    For previously selected metapackages, run the following command:

    ~(keystone_admin)]$ software deploy precheck [-f|--force]

    To specify metapackages directly and reset the current selection, run the following command:

    ~(keystone_admin)]$ software deploy precheck infra_nn.nn

    The following metapackage release(s) will be deployed: infra_nn.nn

    Note

    Each metapackage can implement its own deploy-precheck script. The detailed health checks shown in the failed precheck example below are from the swmgmt metapackage’s deploy-precheck script. Metapackages that do not implement custom checks, such as infra, only produce the generic Running .../deploy-precheck script... and Finished script execution messages.

    Example output for a product release precheck (success):

    ~(keystone_admin)]$ software deploy precheck starlingx-nn.nn
infra_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/infra/support-scripts/deploy-precheck script...
Finished script execution
k8s-v1.34.1_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/k8s-v1.34.1/support-scripts/deploy-precheck script...
Finished script execution
k8s-v1.35.2_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/k8s-v1.35.2/support-scripts/deploy-precheck script...
Finished script execution
swmgmt_nn.nn.0:
Kubernetes upgrade does not block deployment: [OK]
System Health:
All hosts are provisioned: [OK]
All hosts are unlocked/enabled: [OK]
All hosts have vim enabled: [OK]
All hosts have current configurations: [OK]
Ceph Storage Healthy: [OK]
No alarms: [OK]
All kubernetes nodes are ready: [OK]
All kubernetes control plane pods are ready: [OK]
All kubernetes applications are in a valid state: [OK]
All hosts are patch current: [OK]
Active kubernetes version [v1.35.2] is a valid supported version: [OK]
Active controller is controller-0: [OK]
Installed license is valid: [OK]
Valid upgrade path from release 26.10 to nn.nn: [OK]
Required patches are applied: [OK]
Docker filesystem in controllers satisfies the required size of 40GB: [OK]
distcloud_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/distcloud/support-scripts/deploy-precheck script...
Finished script execution
base_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/base/support-scripts/deploy-precheck script...
Finished script execution
k8s-v1.33.0_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/k8s-v1.33.0/support-scripts/deploy-precheck script...
Finished script execution
k8s-v1.32.2_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/k8s-v1.32.2/support-scripts/deploy-precheck script...
Finished script execution
k8s-common_nn.nn:
Running /var/rootdirs/opt/software/releases/nn.nn/k8s-common/support-scripts/deploy-precheck script...
Finished script execution
The following metapackage release(s) will be deployed:
infra_nn.nn, k8s-v1.34.1_nn.nn, k8s-v1.35.2_nn.nn, swmgmt_nn.nn, distcloud_nn.nn, base_nn.nn, k8s-v1.33.0_nn.nn, k8s-v1.32.2_nn.nn, k8s-common_nn.nn

    Example output when a precheck fails (only the unhealthy metapackage details are shown):

    ~(keystone_admin)]$ software deploy precheck StarlingX-nn.nn
Error:
swmgmt_nn.nn:
The following issues have been detected, which prevents deploying the release.
Kubernetes upgrade does not block deployment: [OK]
System Health:
All hosts are provisioned: [OK]
All hosts are unlocked/enabled: [OK]
All hosts have vim enabled: [OK]
All hosts have current configurations: [OK]
Ceph Storage Healthy: [OK]
No alarms: [OK]
All kubernetes nodes are ready: [OK]
All kubernetes control plane pods are ready: [OK]
All kubernetes applications are in a valid state: [OK]
All hosts are patch current: [OK]
Active kubernetes version [v1.35.2] is a valid supported version: [OK]
Active controller is controller-0: [OK]
Installed license is valid: [OK]
Platform Issuer: [Fail]
-> Platform Issuer (system-local-ca) TLS private key is not valid. Only RSA keys are supported.
Please perform the 'Update system-local-ca or Migrate Platform Certificates to use Cert Manager'
procedure to update the Platform Issuer, providing a valid RSA cert/key to be used by the issuer.
Valid upgrade path from release 26.10 to nn.nn: [OK]
Required patches are applied: [OK]
Docker filesystem in controllers satisfies the required size of 40GB: [OK]
The following metapackage release(s) are unhealthy:
swmgmt_nn.nn

    Where:

    --force|-f will ignore non-management affecting alarms. --options|-o snapshot=true will enable health checks for the LVM snapshot feature.

    Note

    The LVM snapshot option on software deploys will result in a snapshot of the main logical volumes, so that in the case of a rollback, the snapshots can be restored in order to speed up the overall rollback procedure. By default, the rollback procedure does not use snapshots. Instead, it rolls back the individual file changes on the main logical volumes.

    Resolve any checks that are not OK and re-run the software deploy precheck command. Use the -f option to ignore non-management affecting alarms.

    Note

    The failed prechecks must be cleared before software deployment is allowed to proceed.

  6. Start the software deployment procedure.

    Note

    • The software deploy start command will automatically run the prechecks of previous steps if the prechecks have not been run or have not passed.

    • The failed prechecks must be cleared before software deployment is allowed to proceed.

    • Configuration cannot be changed during the software deployment process.

    By default, the software deployment procedure cannot be started unless prechecks pass.

    The start command can target a set of metapackages (partial product release) or a full product release. In both cases, the specified metapackages reset any previous selection. If no metapackages are specified, the previously selected metapackages are used:

    For individual or multiple metapackages (resets previous selection):

    ~(keystone_admin)]$ software deploy start [-f|--force] <metapackage-release-1> <metapackage-release-2>

    Or, if no metapackages are specified (uses previously selected metapackages):

    ~(keystone_admin)]$ software deploy start [-f|--force]

    Example for a patch release with a single metapackage:

    ~(keystone_admin)]$ software deploy start StarlingX-nn.nn.n
Selected infra_nn.nn.0 metapackage for deployment
infra_nn.nn.0 is now starting, await for the states: [deploy-start-done | deploy-start-failed] in 'software deploy show'

    Then, monitor the progress of software deploy start using the following commands:

    For a partial metapackage deployment (nested table format):

    ~(keystone_admin)]$ software deploy show

+-------------------------------------------------------+-------+-------------+-------------------+
| Releases                                              | RR    | Pre-Upgrade |       State       |
+-------------------------------------------------------+-------+-------------+-------------------+
| Metapackage    From Release      To Release           | True  |    False    | deploy-start-done |
| -------------  --------------    ----------           |       |             |                   |
| infra          <current-release>  nn.nn               |       |             |                   |
+-------------------------------------------------------+-------+-------------+-------------------+

    Note

    When deploying all metapackages from a product release, the output uses the simple “From Release / To Release” format. The nested table format is only shown for partial metapackage deployments (a subset of the product release).

    The software deploy start command may take 5-10 mins to reach the deploy-start-done state depending on hardware.

    Note

    • If software deploy start fails, that is, if the state is deploy-start-failed, review /var/log/software.log on the active controller for failure details, address the issues, and run the software deploy delete command to delete the deploy and re-execute the software deploy start command.

    • If LVM snapshot feature is enabled on deploy start (AIO-SX support only), the system will take snapshots of the main logical volumes so that these can be used to speed up the rollback by restoring the snapshots, if needed.

  7. Deploy the new software release to all hosts.

    • For an AIO-SX system:

      1. Deploy the new software release to controller-0.

        1. Only if the software deployment is RR=True (reboot required), lock controller-0.

          ~(keystone_admin)]$ system host-lock controller-0

        2. Deploy the new software release to controller-0.

          ~(keystone_admin)]$ software deploy host controller-0
Host installation request sent to controller-0.
Host installation was successful on controller-0.

          After this command completes:

          • If RR=True, the host is still running the old software release, however boot parameters have been updated to boot into the new software release on the next host reboot, which will occur in the next step which unlocks the host.

          • If RR=False, the host is running the new software release.

        3. Only if the software deployment is RR=True, unlock controller-0.

          ~(keystone_admin)]$ system host-unlock controller-0

          The host will now reboot into the new software release. Wait for the host to finish rebooting and become enabled. This may take 3-5 mins depending on hardware.

      2. Proceed to step Activate the software deployment (software deploy activate).

    • For an AIO-DX system or standard system:

      1. Deploy the new software release to controller-1 (standby controller).

        1. Only if the software deployment is RR=True, lock controller-1.

          ~(keystone_admin)]$ system host-lock controller-1

        2. Deploy the new software release to controller-1.

          ~(keystone_admin)]$ software deploy host controller-1
Host installation request sent to controller-1.
Host installation was successful on controller-1.

          After this command completes:

          • If RR=True, the host is still running the old software release, however boot parameters have been updated to boot into the new software release on the next host reboot, which will occur in the next step which unlocks the host.

          • If RR=False, the host is running the new software release.

        3. Only if the software deployment is RR=True, unlock controller-1.

          ~(keystone_admin)]$ system host-unlock controller-1

          The host will now reboot into the new software release. Wait for the host to finish rebooting and become enabled.

          This may take 3-5 mins depending on hardware.

        4. Display state of software deployment.

          Note

          When deploying all metapackages from a product release, the output uses the simple “From Release / To Release” format. When deploying a subset of metapackages, a nested table is shown listing only the metapackages being deployed.

          For a full product release deployment:

          ~(keystone_admin)]$ software deploy show

          Verify that the deployment is transitioning from |prod|-<current-release> to |prod|-nn.nn, that RR is set to True, and that the deployment state is deploy-host.

          ~(keystone_admin)]$ software deploy host-list

          Verify that controller-1 has reached the deploy-host-deployed state, while controller-0, storage-0, storage-1, worker-0, and worker-1 remain in the deploy-host-pending state.

          For a partial metapackage deployment:

          ~(keystone_admin)]$ software deploy show

+------------------------------------------------------+-------+-------------+------------------+
| Releases                                             | RR    | Pre-Upgrade |      State       |
+------------------------------------------------------+-------+-------------+------------------+
| Metapackage    From Release       To Release         | False |    False    | deploy-host      |
| -------------  --------------     ------------       |       |             |                  |
| base           <current-release>  nn.nn              |       |             |                  |
| infra          <current-release>  nn.nn              |       |             |                  |
+------------------------------------------------------+-------+-------------+------------------+

          ~(keystone_admin)]$ software deploy host-list

+--------------+---------------------+-------------------+-------+-------------+---------------------+
| Host         | From Release        | To Release        |  RR   | Pre-Upgrade |        State        |
+--------------+---------------------+-------------------+-------+-------------+---------------------+
| controller-0 | <current-release>   | nn.nn             | False |    False    | deploy-host-pending |
| controller-1 | <current-release>   | nn.nn             | False |    False    | deploy-host-pending |
+--------------+---------------------+-------------------+-------+-------------+---------------------+

      2. Switch the activity from controller-0 such that controller-1 becomes active.

        ~(keystone_admin)]$ system host-swact controller-0

        Wait for the activity to switch to controller-1. This may take up to a minute depending on hardware.

        Reconnect to the system using the floating OAM IP address, as the active controller has changed.

      3. Deploy the new software release to controller-0 (now the standby controller).

        1. Only if the software deployment is RR=True, lock controller-0.

          ~(keystone_admin)]$ system host-lock controller-0

        2. Deploy the new software release to controller-0.

          ~(keystone_admin)]$ software deploy host controller-0
Host installation request sent to controller-0.
Host installation was successful on controller-0.

          After this command completes:

          • If RR=True, the host is still running the old software release, however boot parameters have been updated to boot into the new software release on the next host reboot, which will occur in the next step which unlocks the host.

          • If RR=False, the host is running the new software release.

        3. Only if the software deployment is RR=True, unlock controller-0.

          ~(keystone_admin)]$ system host-unlock controller-0

          The host will now reboot into the new software release. Wait for the host to finish rebooting and become enabled. This may take 3-5 mins depending on hardware.

        4. Verify the deployment state for both controllers.

          For a full product release deployment:

          ~(keystone_admin)]$ software deploy show

          Verify that the deployment is transitioning from |prod|-<current-release> to |prod|-nn.nn, that RR is set to True, and that the deployment state is deploy-host.

          ~(keystone_admin)]$ software deploy host-list

          Verify that controller-0 and controller-1 have reached the deploy-host-deployed state, and that storage-0, storage-1, worker-0, and worker-1 remain in the deploy-host-pending state.

          For a partial metapackage deployment:

          ~(keystone_admin)]$ software deploy show

+----------------------------------------------------+-------+-------------+------------------+
| Releases                                           | RR    | Pre-Upgrade |      State       |
+----------------------------------------------------+-------+-------------+------------------+
| Metapackage    From Release       To Release       | False |    False    | deploy-host      |
| -------------  --------------     ----------       |       |             |                  |
| base           <current-release>  nn.nn            |       |             |                  |
| infra          <current-release>  nn.nn            |       |             |                  |
+----------------------------------------------------+-------+-------------+------------------+

          ~(keystone_admin)]$ software deploy host-list

+--------------+-------------------------+------------------+-------+-------------+----------------------+
| Host         | From Release            | To Release       |  RR   | Pre-Upgrade |        State         |
+--------------+-------------------------+------------------+-------+-------------+----------------------+
| controller-0 | <current-release>       | nn.nn            | False |    False    | deploy-host-deployed |
| controller-1 | <current-release>       | nn.nn            | False |    False    | deploy-host-deployed |
+--------------+-------------------------+------------------+-------+-------------+----------------------+

      4. Check the system health to ensure that there are no unexpected alarms.

        ~(keystone_admin)]$ fm alarm-list

        Clear all the alarms unrelated to the upgrade process.

      5. If storage hosts are present, deploy the new software release to the storage hosts one at a time.

        1. Deploy the new software release to storage-0.

          1. Only if the software deployment is RR=True, lock storage-0.

            ~(keystone_admin)]$ system host-lock storage-0

          2. Deploy the new software release to storage-0.

            ~(keystone_admin)]$ software deploy host storage-0
Host installation request sent to storage-0.
Host installation was successful on storage-0.

            After this command completes:

            • If RR=True, the host is still running the old software release, however boot parameters have been updated to boot into the new software release on the next host reboot, which will occur in the next step which unlocks the host.

            • If RR=False, the host is running the new software release.

          3. Only if the software deployment is RR=True, unlock storage-0.

            ~(keystone_admin)]$ system host-unlock storage-0

            The host will now reboot into the new software release. Wait for the host to finish rebooting and become enabled. Wait for all alarms to clear after the unlock before proceeding to the next storage host. This may take 3-5 mins depending on hardware.

          4. Display state of software deployment.

            For a full product release deployment:

            ~(keystone_admin)]$ software deploy show

            Verify that the deployment is transitioning from |prod|-<current-release> to |prod|-nn.nn, that RR is set to True, and that the deployment state is deploy-host.

            ~(keystone_admin)]$ software deploy host-list

            Verify that controller-0, controller-1, and storage-0 have reached the deploy-host-deployed state, and that storage-1, worker-0, and worker-1 remain in the deploy-host-pending state.

            For a partial metapackage deployment:

            ~(keystone_admin)]$ software deploy show

+-------------------------------------------------------+-------+-------------+------------------+
| Releases                                              | RR    | Pre-Upgrade |      State       |
+-------------------------------------------------------+-------+-------------+------------------+
| Metapackage    From Release       To Release          | False |    False    | deploy-host      |
| -------------  --------------     ----------          |       |             |                  |
| base           <current-release>  nn.nn               |       |             |                  |
| infra          <current-release>  nn.nn               |       |             |                  |
+-------------------------------------------------------+-------+-------------+------------------+

            ~(keystone_admin)]$ software deploy host-list

+--------------+-------------------------+------------------+-------+-------------+----------------------+
| Host         | From Release            | To Release       |  RR   | Pre-Upgrade |        State         |
+--------------+-------------------------+------------------+-------+-------------+----------------------+
| controller-0 | <current-release>       | nn.nn            | False |    False    | deploy-host-deployed |
| storage-0    | <current-release>       | nn.nn            | False |    False    | deploy-host-pending  |
+--------------+-------------------------+------------------+-------+-------------+----------------------+

        2. Repeat the above steps for each storage host.

          Note

          After upgrading the first storage host, you can expect alarm 800.003. The alarm is cleared after all the storage hosts are upgraded.

      6. If worker hosts are present, deploy the new software release to worker hosts one at a time.

        1. Deploy the new software release to worker-0.

          1. Only if the software deployment is RR=True, lock worker-0.

            ~(keystone_admin)]$ system host-lock worker-0

          2. Deploy the new software release to worker-0.

            ~(keystone_admin)]$ software deploy host worker-0
Host installation request sent to worker-0.
Host installation was successful on worker-0.

            After this command completes:

            • If RR=True, the host is still running the old software release, however boot parameters have been updated to boot into the new software release on the next host reboot, which will occur in the next step which unlocks the host.

            • If RR=False, the host is running the new software release.

          3. Only if the software deployment is RR=True, unlock worker-0.

            ~(keystone_admin)]$ system host-unlock worker-0

            The host will now reboot into the new software release. Wait for the host to finish rebooting and become enabled. Wait for all alarms to clear after the unlock before proceeding to the next worker host. This may take 3-5 mins depending on hardware.

          4. Display state of software deployment.

            For a full product release deployment:

            ~(keystone_admin)]$ software deploy show

            Verify that the deployment is transitioning from |prod|-<current-release> to |prod|-nn.nn, that RR is set to True, and that the deployment state is deploy-host.

            ~(keystone_admin)]$ software deploy host-list

            Verify that controller-0, controller-1, storage-0, storage-1, and worker-0 have reached the deploy-host-deployed state, and that worker-1 remains in the deploy-host-pending state.

            For a partial metapackage deployment:

            ~(keystone_admin)]$ software deploy show

+-----------------------------------------------------+-------+-------------+------------------+
| Releases                                            | RR    | Pre-Upgrade |      State       |
+-----------------------------------------------------+-------+-------------+------------------+
| Metapackage    From Release      To Release         | False |    False    | deploy-host      |
| -------------  --------------    ------------       |       |             |                  |
| base           <current-release>  nn.nn             |       |             |                  |
| infra          <current-release>  nn.nn             |       |             |                  |
+-----------------------------------------------------+-------+-------------+------------------+

            ~(keystone_admin)]$ software deploy host-list

+--------------+---------------------------+--------------+-------+-------------+-----------------------+
| Host         | From Release              | To Release   |  RR   | Pre-Upgrade |        State          |
+--------------+---------------------------+--------------+-------+-------------+-----------------------+
| controller-0 | <current-release>         | nn.nn        | False |    False    | deploy-host-deployed  |
| worker-0     | <current-release>         | nn.nn        | False |    False    | deploy-host-pending   |
+--------------+---------------------------+--------------+-------+-------------+-----------------------+

        2. Repeat the above steps for each worker host until the deployment state is deploy-host-done.

          For a full product release deployment:

          ~(keystone_admin)]$ software deploy show

          Verify that the deployment is transitioning from |prod|-<current-release> to |prod|-nn.nn, that RR is set to True, and that the deployment state is deploy-host-done.

          ~(keystone_admin)]$ software deploy host-list

          Verify that controller-0, controller-1, storage-0, storage-1, worker-0, and worker-1 all have reached the deploy-host-deployed state.

          For a partial metapackage deployment:

          ~(keystone_admin)]$ software deploy show

+-----------------------------------------------------+-------+-------------+------------------+
| Releases                                            | RR    | Pre-Upgrade |      State       |
+-----------------------------------------------------+-------+-------------+------------------+
| Metapackage    From Release    To Release           | False |    False    | deploy-host-done |
| -------------  --------------  ------------         |       |             |                  |
| base           <current-release> nn.nn              |       |             |                  |
| infra          <current-release> nn.nn              |       |             |                  |
+-----------------------------------------------------+-------+-------------+------------------+

          ~(keystone_admin)]$ software deploy host-list

+--------------+--------------------------+------------+-------+-------------+----------------------+
| Host         | From Release             | To Release |  RR   | Pre-Upgrade |        State         |
+--------------+--------------------------+------------+-------+-------------+----------------------+
| controller-0 | <current-release>        | nn.nn      | False |    False    | deploy-host-deployed |
| controller-1 | <current-release>        | nn.nn      | False |    False    | deploy-host-deployed |
| worker-0     | <current-release>        | nn.nn      | False |    False    | deploy-host-deployed |
| worker-1     | <current-release>        | nn.nn      | False |    False    | deploy-host-deployed |
+--------------+--------------------------+------------+-------+-------------+----------------------+

      7. Switch the activity from controller-1 such that controller-0 becomes active.

        ~(keystone_admin)]$ system host-swact controller-1

        Wait for the activity to switch to controller-0. This may take up to a minute depending on hardware.

        Reconnect to the system using the floating OAM IP address, as the active controller has changed.

  8. Activate the software deployment.

    ~(keystone_admin)]$ software deploy activate
Deploy activate has started

    When running the software deploy activate command, new configurations are applied to the controller. The 250.001 (Configuration is out-of-date) alarms are raised and are cleared as the configurations are applied.

    The software deployment state goes from deploy-activate to deploy-activate-done once deployment is activated. For a major release software deployment, this may take up to 15-30 mins to complete depending on system configuration and hardware.

    For a full product release deployment:

    ~(keystone_admin)]$ software deploy show

    Verify that the output shows the deployment transitioning from StarlingX-<current-release> to StarlingX-nn.nn with RR=True and a state of deploy-activate-done.

    For a partial metapackage deployment:

    ~(keystone_admin)]$ software deploy show

+--------------------------------------------------------+-------+-------------+----------------------+
| Releases                                               | RR    | Pre-Upgrade |        State         |
+--------------------------------------------------------+-------+-------------+----------------------+
| Metapackage    From Release      To Release            | False |    False    | deploy-activate-done |
| -------------  --------------    ------------          |       |             |                      |
| base           <current-release>  nn.nn                |       |             |                      |
| infra          <current-release>  nn.nn                |       |             |                      |
+--------------------------------------------------------+-------+-------------+----------------------+

    Note

    If software deploy activate fails, that is, if the state is deploy-activate-failed, review /var/log/software.log on the active controller for failure details, address the issues, and re-execute the software deploy activate command.

  9. Complete the software deployment.

    ~(keystone_admin)]$ software deploy complete
Deployment has been completed

    For a full product release deployment:

    ~(keystone_admin)]$ software deploy show

    Verify that the output shows the deployment transitioning from StarlingX-<current-release> to StarlingX-nn.nn with RR=True and a state of deploy-completed.

    For a partial metapackage deployment:

    ~(keystone_admin)]$ software deploy show

+----------------------------------------------------------+-------+-------------+------------------+
| Releases                                                 | RR    | Pre-Upgrade |      State       |
+----------------------------------------------------------+-------+-------------+------------------+
| Metapackage    From Release       To Release             | False |    False    | deploy-completed |
| -------------  --------------     ------------           |       |             |                  |
| base           <current-release>  nn.nn                  |       |             |                  |
| infra          <current-release>  nn.nn                  |       |             |                  |
+----------------------------------------------------------+-------+-------------+------------------+

    Note

    After this command is executed, you can run the Kubernetes version upgrade procedure, if desired to upversion to new Kubernetes versions available in the new software release.

  10. Delete the software deployment.

    Note

    For a major release deployment, after this command is executed, the major release software deployment cannot be rolled back.

    ~(keystone_admin)]$ software deploy delete
Deployment has been deleted

    ~(keystone_admin)]$ software deploy show
No deploy in progress

Results

The software deployment is complete. All hosts are running the new software release and the deployment state shows no deploy in progress. The software list confirms the new release is in the deployed state.

Note

After the deploy delete, if there are previous release entries in the unavailable state, the alarm 900.024 Obsolete release in system is raised.

Postrequisites

Note

Do not delete any previous release(s) from the System Controller while there are subclouds running those release(s).

In the case of software deployment of a new major release, you should remove the old major release to reclaim disk space.

~(keystone_admin)]$ software list

The output of software list shows StarlingX-<current-release> in the unavailable state and StarlingX-nn.nn in the deployed state.

~(keystone_admin)]$ software delete StarlingX-<current-release>
StarlingX-<current-release> has been deleted.

~(keystone_admin)]$ software list

After deletion, the output of software list shows only StarlingX-nn.nn in the deployed state.