Handle Kubernetes Version Upgrade Orchestration Failures¶
The creation or application of a strategy could fail for any of the listed reasons described in this section. Follow the suggested actions in each case to resolve the issue.
Strategy creation failure¶
Reason: Build failed with no reason.
Action:
Verify that the
--worker-apply-type
was not set toignore
.Check recent logs added to /var/log/nfv-vim.log.
Reason: Alarms from platform are present.
Action:
Query for management affecting alarms and take actions to clear them.
~(keystone_admin)$ fm alarm-list --mgmt_affecting
If there are no management affecting alarms present, take actions to clear other reported alarms or try creating the strategy with the
relaxed
alarms restrictions option--alarm-restrictions relaxed
.
Reason: No Kubernetes version upgrade required.
Action:
Verify that the Kubernetes patches have been uploaded and applied. Verify the version of Kubernetes on the hosts by executing “system kube-host-upgrade-list.
Note
If the strategy create failed, first you must resolve it. You must delete the failed strategy before you create another strategy.
Strategy Apply Failure¶
Reason: Alarms from platform are present.
Action: This suggests that an alarm has been raised since the creation of the strategy. Address the cause of the new alarm, delete the strategy and try creating and applying a new strategy.
Reason: Unable to migrate instances.
Action: See Kubernetes Version Upgrade Operations Requiring Manual Migration for steps to resolve migration issues.
Reason: Kubernetes version upgrade failed. Suggests that the Kubernetes upgrade for the specified host has failed.
Action: Consult the StarlingX community.
Reason: Lock host failed.
Action:
Investigate the
/var/log/sysinv.log
, and/var/log/nfv-vim.log
files.Address the underlying issue.
Manually lock and unlock the host.
Try recreating and re-applying the Kubernetes version upgrade strategy to automatically finish the upgrade process.
Reason: Unlock host failed.
Action:
Investigate
/var/log/mtcAgent.log
file for cause logs files.Address the underlying issue.
Manually lock and unlock the host to recover.
Try recreating and re-applying the Kubernetes version upgrade strategy to automatically finish the upgrade process.
Note
If the strategy apply fails, you must resolve the strategy:command:apply failure, and delete the failed strategy before trying to create and apply another strategy.