Nova Ephemeral Storage

Ephemeral storage for VMs, which includes swap disk storage, ephemeral disk storage, and root disk storage if the VM is configured for boot-from-image, is implemented by the nova service.

For flexibility and scalability, stx-openstack supports four configuration options to store Nova ephemeral disks. Operators can configure the Nova deployment using the storage_conf option in the Nova chart.

Platform Host Path

The storage on compute hosts is used for the following:

• the nova image cache, containing images downloaded from Glance

• various small nova control and log files, such as the libvirt.xml file, which is used to pass parameters to libvirt at launch, and the console.log file

This configuration uses a host path directly and is possible in two mutually exclusive options: a platform local disk or a platform dedicated file system.

• nova-local volume group:

  This option provides the best disk IO performance. The nova-local group can be backed locally by one or more disks or partitions on the compute host. For more information about nova-local backing options, see StarlingX Storage Configuration and Management: Block Storage for Virtual Machines.

  # Create the nova-local Logical Volume Group for the host
  system host-lvg-add controller-1 nova-local
  
  # List disks on this host;
  # find a disk NOT being used as the system disk and NOT being used as for ceph,
  # and get this free disk’s UUID
  system host-disk-list controller-1                            # lists all disks
  system host-show controller-1 | fgrep rootfs                  # lists system disk
  system host-stor-list controller-1                            # lists all ceph disks
  
  # Add this disk to the nova-local Logical Volume Group
  system host-pv-add controller-1 nova-local ${FREEDISKUUID}
  

  In cases where not all hosts in the system are configured in this way, you can use a host aggregate and an aggregate_instance_extra_specs property in the VM flavor to ensure that the VM is scheduled on hosts with local storage enabled for nova ephemeral.

• instances file system (NOT RECOMMENDED):

  Use available space on the local system disk, by configuring space for the instances filesystem. This option is the simplest, but NOT recommended as the disk IO performance of the ephemeral disk of VMs is impacted by platform use of the system disk, and vice versa.

  system host-fs-add controller-1 instances=30  # 30G of disk space
  

For CoW-image-backed local storage, Instances LV is also used to hold CoW-image disk files for use as VM disk storage. It is the only volume in nova-local.

By default, no size is specified for the Instances LV. For non-root disks, the minimum required space is 2 GB for a nova-local volume group with a total size less that 80 GB, and 5 GB for a nova-local volume group larger or equal than 80 GB; you must specify at least this amount. You can allocate more Instances LV space to support the anticipated number of boot-from-image VMs, up to 50% of the maximum available storage of the local volume group. At least 50% free space in the volume group is required to provide space for allocating logical volume disks for launched instances. The value provided for the Instance LV Size is limited by this maximum.

Instructions for allocating the Instances LV Size using the Web administration interface or the CLI are included as part of configuring the compute nodes. Suggested sizes are indicated in the Web administration interface.

Caution

If less than the minimum required space is available, the compute host cannot be unlocked.

Once one of the two mutually exclusive options is configured in the host, stx-openstack can be configured:

• Create a nova.yaml file with the desired configuration overrides:

  storage_conf:
    volume_storage_class_priority:
      - host-path
  

• Initialize the platform environment:

  $ source /etc/platform/openrc
  

• Apply the Nova configuration overrides:

  $ system helm-override-update --reuse-values --values nova.yaml |prefix|-openstack nova openstack
  

  Use the --reuse-values option whenever you update chart overrides to preserve any previously configured values.

For more information on managing the nova-local volume group or the instances filesystem, see:

• Create or Change the Size of Nova-local Storage

• Replace Nova-local Storage Disk on a Simplex System

• Replace a Nova-Local Disk

• Specify the Storage Type for VM Ephemeral Disks

Ceph Pool

This option uses StarlingX rook-ceph storage, which provides better live migration performance because ephemeral disks are remote and do not need to be copied between hosts. Additionally, VMs with ephemeral disks backed by Ceph retain their ephemeral disks if the compute node fails and the VM is restarted on another compute node.

To enable Ceph-based storage for Nova ephemeral disks, the following prerequisites must be met:

• A rook-ceph cluster is already configured on the system.

• The host has a Ceph OSD configured.

• Neither the nova-local volume group nor the instances filesystem is configured.

Then, configure stx-openstack accordingly:

• Create a nova.yaml file with the desired configuration overrides:

  storage_conf:
    volume_storage_class_priority:
      - ceph
  

• Initialize the platform environment:

  $ source /etc/platform/openrc
  

• Apply the Nova configuration overrides:

  $ system helm-override-update --reuse-values --values nova.yaml |prefix|-openstack nova openstack
  

  Use the --reuse-values option whenever you update chart overrides to preserve any previously configured values.

When this option is configured, stx-openstack automatically creates a Ceph pool named ephemeral and uses the RBD protocol to store all ephemeral disks on it.

NFS Mount

This option uses a NFS mount to store ephemeral disks. The mount is pre-configured by the operator on the host and can use any storage provider. stx-openstack mounts the NFS shares directly into its containers via Kubernetes VolumeMounts with NFS native support.

Because ephemeral disks are remote, this option provides better live migration performance, as disks do not need to be copied between hosts. VMs with ephemeral disks backed by NFS also retain their ephemeral disks if the compute node fails and the VM is restarted on another compute node.

stx-openstack can be configured accordingly by:

• Creating a nova.yaml file with the desired configuration overrides:

  storage_conf:
    volume_storage_class_priority:
      - nfs
    nfs_shares:
      server: 127.0.0.1
      path: /local/path/to/openstack_instances
  

• Initialize the platform environment:

  $ source /etc/platform/openrc
  

• Apply the Nova configuration overrides:

  $ system helm-override-update --reuse-values --values nova.yaml |prefix|-openstack nova openstack
  

  Use the --reuse-values option whenever you update chart overrides to preserve any previously configured values.

See NetApp Storage Configuration for OpenStack for reference on NetApp backed NFS configuration.

Kubernetes Volumes

This option is not recommended for production, but is available for proof-of-concept validations. It uses a PVC with StorageClass backed by any given remote storage (NetApp or Extended Storage Backends)

It requires that a remote storage backend is already configured on the system. See NetApp Storage Configuration for OpenStack or Select the Backend for Other Services for details.