Roll Back a Software Upgrade After the Second Controller Upgrade

After the second controller is upgraded, you can still roll back a software upgrade, however, the rollback will impact the hosting of applications.

The upgrade abort procedure can only be applied before the upgrade-complete command is issued. Once this command is issued the upgrade cannot be aborted. If you must revert to the previous release, then restore the system using the backup data taken prior to the upgrade.

In some scenarios additional actions will be required to complete the upgrade abort. It may be necessary to restore the system from a backup.

Procedure

  1. Run the upgrade-abort command to abort the upgrade.

    ~(keystone_admin)]$ system upgrade-abort

    Once this is done there is no going back; the upgrade must be completely aborted.

    The following state applies when you run this command.

    • aborting-reinstall:

      • State entered when system upgrade-abort is executed after upgrading controller-0.

      • Remain in this state until the abort is completed.

  2. Make controller-1 active.

    ~(keystone_admin)]$ system host-swact controller-0

  3. Lock controller-0.

    ~(keystone_admin)]$ system host-lock controller-0

  4. Wipe the disk and power down all storage (if applicable) and worker hosts.

    Note

    Skip this step if doing this procedure on a StarlingX Duplex system.

    1. Execute wipedisk from the shell on each storage or worker host.

    2. Power down each host.

  5. Lock all storage (if applicable) and worker hosts.

    Note

    Skip this step if doing this procedure on a StarlingX Duplex system.

    ~(keystone_admin)]$ system host-lock <hostID>

  6. Downgrade controller-0.

    ~(keystone_admin)]$ system host-downgrade controller-0

    The host is re-installed with the previous release load.

  7. Unlock controller-0.

    ~(keystone_admin)]$ system host-unlock controller-0

    Note

    Wait for controller-0 to become unlocked-enabled. Wait for the DRBD sync 400.001 Services-related alarm to be raised and then cleared.

  8. Swact to controller-0.

    ~(keystone_admin)]$ system host-swact controller-1

    Swacting back to controller-0 will switch back to using the previous release databases, which were frozen at the time of the swact to controller-1. This is essentially the same result as a system restore.

  9. Lock and downgrade controller-1.

    ~(keystone_admin)]$ system host-lock controller-1

    ~(keystone_admin)]$ system host-downgrade controller-1

    The host is re-installed with the previous release load.

  10. Unlock controller-1.

    ~(keystone_admin)]$ system host-unlock controller-1

  11. Power up and unlock the storage hosts one at a time (if using a Ceph storage backend). The hosts are re-installed with the previous release load.

    Note

    Skip this step if doing this procedure on a StarlingX Duplex system.

  12. Power up and unlock the worker hosts one at a time.

    Note

    Skip this step if doing this procedure on a StarlingX Duplex system.

    The hosts are re-installed with the previous release load. As each worker host goes online, application pods will be automatically recovered by the system.

  13. Complete the upgrade.

    ~(keystone_admin)]$ system upgrade-complete

    This cleans up the upgrade release, configuration, databases, and so forth.

  14. Delete the upgrade release load.

    ~(keystone_admin)]$ system load-delete