LVM CSI Application Overview¶
The LVM CSI application provides seamless integration of TopoLVM into the StarlingX system, allowing users to leverage TopoLVM functionalities with StarlingX system automations.
TopoLVM is a Kubernetes CSI driver (storage plugin) for the Linux LVM tool that provides flexibility in volume management, an enhanced scheduler for large storage capacity, and dynamic provisioning for local storage. TopoLVM extends the standard Kubernetes scheduler by providing a customized, more dynamic tool with a topology-aware approach.
For additional information, consult the TopoLVM documentation: https://github.com/topolvm/topolvm
Warning
TopoLVM is a plugin designed to provide a local storage system. TopoLVM resources are valid only on the nodes to which they were assigned. On a multi-node cluster, high availability features are not expected for TopoLVM resources.
Configuration Summary¶
The LVM CSI application can be configured by following these steps:
For each node that will support LVM operations, add a volume group (VG):
If the node has only one disk, use CGTS-VG as the standard VG.
If the node has additional disks, use a dedicated disk and create a VG for LVM CSI application purposes.
Add the LVM storage backend.
Wait for the application to be automatically applied.
Create your application’s PVCs using one of the available LVM storage classes.
Minimum Configuration Path¶
This section describes how to apply the LVM CSI application using the minimum required configuration:
An LVM storage backend added.
At least one VG configured with the
lvm-csifunction.
Add LVM CSI support on the CGTS-VG volume group (root disk).
~(keystone_admin)]$ system host-lvg-modify controller-0 cgts-vg -f lvm-csi +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | lvm_vg_name | cgts-vg | | vg_state | provisioned | | uuid | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | lvm_vg_access | wz--n- | | lvm_max_lv | 0 | | lvm_cur_lv | 13 | | lvm_max_pv | 0 | | lvm_cur_pv | 1 | | lvm_vg_size_gib | 261.375 | | lvm_vg_avail_size_gib | 68.562 | | lvm_vg_total_pe | 66912 | | lvm_vg_free_pe | 17552 | | lvm_function | lvm-csi | | lvm_type | thin | | lvm_pool_size | 34 | | created_at | 2026-07-31T10:17:55.470587+00:00 | | updated_at | 2026-07-31T17:10:36.479284+00:00 | | parameters | {'thin_cur_lv': 0} | +-----------------------+--------------------------------------+
Validate the configuration.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 34 | wz--n- | 261.375 | 34.492 | 1 | 13 | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Add the LVM storage backend.
~(keystone_admin)]$ system storage-backend-add lvm System configuration has changed. Please follow the administrator guide to complete configuring the system. +--------------------------------------+------------+---------+----------------------+----------+----------+------------------------------------------------+ | uuid | name | backend | state | task | services | capabilities | +--------------------------------------+------------+---------+----------------------+----------+----------+------------------------------------------------+ | c90c0c67-20cf-4fdc-9a31-f6be96fe9172 | lvm-store | lvm | configuring-with-app | uploaded | None | | +--------------------------------------+------------+---------+----------------------+----------+----------+------------------------------------------------+
Wait for the LVM CSI application to reach
appliedstatus.~(keystone_admin)]$ system application-list +--------------------------+-----------+-------------------------------------------+------------------+----------+----------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +--------------------------+-----------+-------------------------------------------+------------------+----------+----------------------------------------------+ | lvm-csi | 26.10-0 | lvm-csi-fluxcd-manifests | fluxcd-manifests | applying | Applying app lvm-csi. Overall completion: 0% | +--------------------------+-----------+-------------------------------------------+------------------+----------+----------------------------------------------+ ----- | lvm-csi | 26.10-0 | lvm-csi-fluxcd-manifests | fluxcd-manifests | applied | completed |
Validate the storage class existence.
~(keystone_admin)]$ kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE cgts-vg topolvm.io Delete WaitForFirstConsumer true 3m6s
Apply the resource.
~(keystone_admin)]$ kubectl apply -f pod.yaml persistentvolumeclaim/my-pvc1 created persistentvolumeclaim/my-pvc2 created pod/my-pod created
Example of PVC:
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: my-pvc1 spec: accessModes: - ReadWriteOnce resources: requests: storage: 256Mi storageClassName: cgts-vg
Validate the pod and PVCs.
~(keystone_admin)]$ kubectl get pods NAME READY STATUS RESTARTS AGE my-pod 1/1 Running 0 33s ~(keystone_admin)]$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE my-pvc1 Bound pvc-c4bff00b-9032-40fb-8a02-45e250b19f48 300Mi RWO cgts-vg <unset> 37s my-pvc2 Bound pvc-11ae0e50-4c21-4513-a150-3f4253fae07c 300Mi RWO cgts-vg <unset> 37s
LVM Configuration Actions¶
This section describes how to manage volume groups (VGs) and logical volumes (LVs) for LVM CSI application purposes.
Add Volume Groups for LVM CSI Purposes¶
Use Only the Root Disk¶
If the cluster nodes have only a single disk (the root disk), the free space can be used to support LVM CSI application operations. This strategy only supports thin provisioning.
Add the lvm-csi Function to CGTS-VG¶
To use LVM resources on the root disk, follow the steps below.
Option 1: Default thin pool size
Add the
lvm-csifunction to the CGTS-VG volume group using the host-lvg-modify command.The modify command does not allow changing the provisioning type, so the only supported type is thin provisioning.
When CGTS-VG is configured with the
lvm-csifunction, the thin pool is created using half of the available VG space.~(keystone_admin)]$ system host-lvg-modify controller-0 cgts-vg -f lvm-csi +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | lvm_vg_name | cgts-vg | | vg_state | provisioned | | uuid | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | lvm_vg_access | wz--n- | | lvm_max_lv | 0 | | lvm_cur_lv | 13 | | lvm_max_pv | 0 | | lvm_cur_pv | 1 | | lvm_vg_size_gib | 261.375 | | lvm_vg_avail_size_gib | 68.562 | | lvm_vg_total_pe | 66912 | | lvm_vg_free_pe | 17552 | | lvm_function | lvm-csi | | lvm_type | thin | | lvm_pool_size | 34 | | created_at | 2026-07-31T10:17:55.470587+00:00 | | updated_at | 2026-07-31T16:13:49.513551+00:00 | | parameters | {'thin_cur_lv': 0} | +-----------------------+--------------------------------------+
Option 2: Set function and thin pool size
Add the
lvm-csifunction and specify the thin pool size for the CGTS-VG volume group using the host-lvg-modify command.The thin pool size (passed with the
-soption) must be an integer value in GB. It must be greater than 1 GB and must leave at least 1 GB free in the VG.~(keystone_admin)]$ system host-lvg-modify controller-0 cgts-vg -f lvm-csi -s 40 +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | lvm_vg_name | cgts-vg | | vg_state | provisioned | | uuid | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | lvm_vg_access | wz--n- | | lvm_max_lv | 0 | | lvm_cur_lv | 13 | | lvm_max_pv | 0 | | lvm_cur_pv | 1 | | lvm_vg_size_gib | 261.375 | | lvm_vg_avail_size_gib | 34.492 | | lvm_vg_total_pe | 66912 | | lvm_vg_free_pe | 8830 | | lvm_function | lvm-csi | | lvm_type | thin | | lvm_pool_size | 40 | | created_at | 2026-07-31T10:17:55.470587+00:00 | | updated_at | 2026-07-31T16:14:17.800642+00:00 | | parameters | {'thin_cur_lv': 0} | +-----------------------+--------------------------------------+
List the available VGs and validate the configuration.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 34 | wz--n- | 261.375 | 34.492 | 1 | 13 | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Resize the Thin Pool on CGTS-VG¶
After the lvm-csi function is assigned, the thin pool size can be
increased. However, use caution with resizing operations. StarlingX does not
support LVM data-move operations by default. After extending the size of a
thin pool, the operation cannot be undone.
List the available VGs and their sizes.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 40 | wz--n- | 261.375 | 28.484 | 1 | 13 | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Change the size using the
-sargument followed by an integer value representing the desired thin pool size. The new size must leave at least 1 GB of free space in the VG.~(keystone_admin)]$ system host-lvg-modify controller-0 cgts-vg -s 42 +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | lvm_vg_name | cgts-vg | | vg_state | provisioned | | uuid | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | lvm_vg_access | wz--n- | | lvm_max_lv | 0 | | lvm_cur_lv | 13 | | lvm_max_pv | 0 | | lvm_cur_pv | 1 | | lvm_vg_size_gib | 261.375 | | lvm_vg_avail_size_gib | 28.484 | | lvm_vg_total_pe | 66912 | | lvm_vg_free_pe | 7292 | | lvm_function | lvm-csi | | lvm_type | thin | | lvm_pool_size | 42 | | created_at | 2026-07-31T10:17:55.470587+00:00 | | updated_at | 2026-07-31T16:15:23.827720+00:00 | | parameters | {'thin_cur_lv': 0} | +-----------------------+--------------------------------------+
Verify the new thin pool size with the host-lvg-list command.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 42 | wz--n- | 261.375 | 26.476 | 1 | 13 | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Command Prototype¶
usage: system host-lvg-modify [-f <lvm_function>] [-s <lvm_pool_size>] <hostname or id> <lvg name or uuid>
Modify the attributes of a Local Volume Group.
Positional arguments:
<hostname or id> Name or ID of the host [REQUIRED]
<lvg name or uuid> Name or UUID of lvg [REQUIRED]
Optional arguments:
-f <lvm_function>, --lvm_function <lvm_function>
Optional function of the Volume Group. Possible values [lvm-csi, none]
-s <lvm_pool_size>, --lvm_pool_size <lvm_pool_size>
Determines the maximum size of the thin pool when configured in cgts-vg. Value must be an integer in GB.
Use Dedicated Disks¶
Adding a dedicated disk as LVM storage is part of a multi-disk strategy for storage deployment. This strategy requires a free disk or partition available on the system.
During configuration, you can select the name of the VG. Any name that meets the function requirements is acceptable. However, use the same name and the same provisioning type on all nodes that will host the LVM CSI application; otherwise, the application apply will fail.
In this strategy, the entire disk space is used and resizing is not supported.
Repeat the steps below for each node that will support LVM operations.
Note
It is suggested to execute these commands with the host in online/unlocked state.
List the disks on the node.
~(keystone_admin)]$ system host-disk-list controller-0 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------------------+--------------------------------------------+ | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path | | | de | num | type | gib | gib | | | | +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------------------+--------------------------------------------+ | 855d0118-4135-4948-a08f-0d1f652ef8e6 | /dev/sda | 2048 | HDD | 292. | 0.0 | Undetermined | VB6eb6551b-121ab29f | /dev/disk/by-path/pci-0000:00:0d.0-ata-1.0 | | | | | | 968 | | | | | | c14f2e6a-4dd3-4492-9ceb-34bc61c15575 | /dev/sdb | 2064 | HDD | 9.765 | 9.761 | Undetermined | VBcaf23f2b-73330c7b | /dev/disk/by-path/pci-0000:00:0d.0-ata-2.0 | | 420aca8d-0b33-4978-bfdb-184a7e47fd25 | /dev/sdc | 2080 | HDD | 9.765 | 0.0 | Undetermined | VB7ae3f12c-78a5030a | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0 | +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------------------+--------------------------------------------+
Add a new VG using the host-lvg-add command.
For LVM CSI application use, you must assign the
lvm-csifunction to the VG and configure the provisioning type. If the type is not specified, thick provisioning is selected by default.~(keystone_admin)]$ system host-lvg-add controller-0 lvm-provisioner -f lvm-csi +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | lvm_vg_name | lvm-provisioner | | vg_state | adding | | uuid | fc8bbd7e-1aec-4da1-9575-952836ab82fc | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | lvm_vg_access | None | | lvm_max_lv | 0 | | lvm_cur_lv | 0 | | lvm_max_pv | 0 | | lvm_cur_pv | 0 | | lvm_vg_size_gib | 0.0 | | lvm_vg_avail_size_gib | 0.0 | | lvm_vg_total_pe | 0 | | lvm_vg_free_pe | 0 | | lvm_function | lvm-csi | | lvm_type | thick | | lvm_pool_size | None | | created_at | 2026-07-31T16:18:34.367208+00:00 | | updated_at | None | | parameters | {} | +-----------------------+--------------------------------------+
Assign a physical volume (PV) to the new VG using the host-pv-add command.
Always use the device path instead of the device node to identify the disk. The device node can change during node reboot or discovery, leading to misconfiguration.
~(keystone_admin)]$ system host-pv-add controller-0 lvm-provisioner /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0 +--------------------------+--------------------------------------------+ | Property | Value | +--------------------------+--------------------------------------------+ | uuid | bb08235f-19cc-45dd-9cd8-c8a3fe246ad8 | | pv_state | adding | | pv_type | disk | | disk_or_part_uuid | 420aca8d-0b33-4978-bfdb-184a7e47fd25 | | disk_or_part_device_node | /dev/sdc | | disk_or_part_device_path | /dev/disk/by-path/pci-0000:00:0d.0-ata-3.0 | | lvm_pv_name | /dev/sdc | | lvm_vg_name | lvm-provisioner | | lvm_pv_uuid | None | | lvm_pv_size_gib | 0.0 | | lvm_pe_total | 0 | | lvm_pe_alloced | 0 | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | created_at | 2026-07-31T16:19:20.066001+00:00 | | updated_at | None | +--------------------------+--------------------------------------------+
Validate the configuration.
Note
During configuration, the new VG may temporarily appear in “provisioning” state. This is expected behavior.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 42 | wz--n- | 261.375 | 26.476 | 1 | 13 | | fc8bbd7e-1aec-4da1-9575-952836ab82fc | lvm-provisioner | provisioned | lvm-csi | thick | None | wz--n- | 9.761 | 9.761 | 1 | 0 | +--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+
Command Prototype¶
~(keystone_admin)]$ system help host-lvg-add
usage: system host-lvg-add [-f <lvm_function>] [-t <lvm_type>] <hostname or id> <lvg name>
Add a Local Volume Group.
Positional arguments:
<hostname or id> Name or ID of host [REQUIRED]
<lvg name> Name of the Local Volume Group [REQUIRED]
Optional arguments:
-f <lvm_function>, --lvm_function <lvm_function>
Optional function of the Volume Group. The possible value is [lvm-csi]
-t <lvm_type>, --lvm_type <lvm_type>
Determines the thick or thin provisioning format of the LVM volume group. Possible values: [thin, thick]. Default: [thick]
Add the Storage Backend¶
Adding the LVM storage backend is required before any LVM CSI application resources can be provisioned.
The LVM storage backend can coexist with any other backend, such as Ceph Rook or Ceph bare-metal.
List the storage backends and confirm that LVM is not already configured.
~(keystone_admin)]$ system storage-backend-list +--------------------------------------+------------+---------+------------+------+----------+--------------------------------------------------------------------------------------------+ | uuid | name | backend | state | task | services | capabilities | +--------------------------------------+------------+---------+------------+------+----------+--------------------------------------------------------------------------------------------+ | 7dfa7f92-1334-44a1-a608-57e19c82a357 | ceph-store | ceph | configured | None | None | replication: 2 min_replication: 1 | +--------------------------------------+------------+---------+------------+------+----------+--------------------------------------------------------------------------------------------+
The system will reject any attempt to add a duplicate storage backend.
Add the LVM storage backend using the storage-backend-add command.
~(keystone_admin)]$ system storage-backend-add lvm System configuration has changed. Please follow the administrator guide to complete configuring the system. +--------------------------------------+------------+---------+----------------------+----------+----------+------------------------------------------------+ | uuid | name | backend | state | task | services | capabilities | +--------------------------------------+------------+---------+----------------------+----------+----------+------------------------------------------------+ | 7dfa7f92-1334-44a1-a608-57e19c82a357 | ceph-store | ceph | configured | None | None | replication: 2 min_replication: 1 | | c90c0c67-20cf-4fdc-9a31-f6be96fe9172 | lvm-store | lvm | configuring-with-app | uploaded | None | | +--------------------------------------+------------+---------+----------------------+----------+----------+------------------------------------------------+
The LVM CSI application is a managed application, which means it will be automatically applied once the minimum requirements are met.
Check Application Auto-Apply Status and Overrides¶
After system configuration, all VGs are automatically discovered by the
application and an override is created to reflect this configuration. All nodes
with a VG configured with the lvm-csi function will receive the
lvm-csi=enabled label, allowing TopoLVM pod deployment.
Override Structure¶
The override follows the pattern below:
- storageClasses
For each VG with the
lvm-csifunction assigned, a storage class with the same name is created. At least one storage class will be present.- lvmd - deviceClasses
This structure defines the configuration for logical volume (LV) creation, such as the spare size.
In thin provisioning mode, the device class is configured with an over-provisioning ratio (1 for CGTS-VG and 2 for other VGs). More than one device class can be configured per storage class.
The over-provisioning parameter sets a multiplier that enables LVM to allocate more space than the volume group’s actual available capacity. It is not recommended to change the overprovisioning ratio for CGTS-VG.
- lvmd - lvcreateOptionClasses
This structure adds a label used to tag LVs created by TopoLVM.
Example of a standard override for the LVM CSI application:
~(keystone_admin)]$ system helm-override-show lvm-csi lvm-csi topolvm-system
+--------------------+-------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------+-------------------------------------------------------------------------------------------------------+
| attributes | enabled: true |
| | |
| combined_overrides | controller: |
| | replicaCount: 1 |
| | lvmd: |
| | deviceClasses: |
| | - name: cgts-vg-thin |
| | spare-gb: 1 |
| | thin-pool: |
| | name: lvmcsi-pool |
| | overprovision-ratio: 1 |
| | type: thin |
| | volume-group: cgts-vg |
| | - name: lvm-provisioner-thin |
| | spare-gb: 1 |
| | thin-pool: |
| | name: lvmcsi-pool |
| | overprovision-ratio: 2 |
| | type: thin |
| | volume-group: lvm-provisioner |
| | lvcreateOptionClasses: |
| | - name: cgts-vg-thin |
| | options: |
| | - --addtag=topolvm |
| | - name: lvm-provisioner-thin |
| | options: |
| | - --addtag=topolvm |
| | storageClasses: |
| | - name: cgts-vg |
| | storageClass: |
| | allowVolumeExpansion: true |
| | fsType: xfs |
| | isDefaultClass: false |
| | volumeBindingMode: WaitForFirstConsumer |
| | - name: lvm-provisioner |
| | storageClass: |
| | allowVolumeExpansion: true |
| | fsType: xfs |
| | isDefaultClass: false |
| | volumeBindingMode: WaitForFirstConsumer |
| | |
| name | lvm-csi |
| namespace | topolvm-system |
+--------------------+-------------------------------------------------------------------------------------------------------+
Auto-Apply Requirements¶
The LVM CSI application is a managed application, which means it remains in
uploaded state until the minimum requirements are met. The application is
automatically applied when:
The LVM storage backend has been added.
At least one VG with the
lvm-csifunction is in “provisioned” state.All nodes with a
lvm-csiVG share the same VG names.
After the application reaches applied status, the following pods are
expected: controller pods (only on controller nodes), lvmd and node pods (on
each configured node), and scheduler pods (on each controller for non-AIO-SX
clusters).
~(keystone_admin)]$ kubectl get pods -n topolvm-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
lvm-csi-topolvm-controller-577d6c6c8d-r2f4t 5/5 Running 0 3h25m 172.16.166.155 controller-1 <none> <none>
lvm-csi-topolvm-lvmd-0-69xzw 1/1 Running 0 3h25m 172.16.192.92 controller-0 <none> <none>
lvm-csi-topolvm-lvmd-0-9d2t5 1/1 Running 0 3h25m 172.16.154.5 compute-0 <none> <none>
lvm-csi-topolvm-lvmd-0-ktl9t 1/1 Running 0 3h25m 172.16.166.153 controller-1 <none> <none>
lvm-csi-topolvm-node-6627n 3/3 Running 0 3h25m 172.16.166.154 controller-1 <none> <none>
lvm-csi-topolvm-node-hzvmv 3/3 Running 0 3h25m 172.16.192.82 controller-0 <none> <none>
lvm-csi-topolvm-node-kstfq 3/3 Running 1 (3h25m ago) 3h25m 172.16.154.4 compute-0 <none> <none>
lvm-csi-topolvm-scheduler-87ndf 1/1 Running 0 3h25m 192.168.206.3 controller-1 <none> <none>
lvm-csi-topolvm-scheduler-f6kkq 1/1 Running 0 3h25m 192.168.206.2 controller-0 <none> <none>
Validate the presence of the storage classes. Each VG should be reflected as a storage class.
~(keystone_admin)]$ kubectl get storageclasses.storage.k8s.io
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
cephfs cephfs.csi.ceph.com Delete Immediate true 68m
cgts-vg topolvm.io Delete WaitForFirstConsumer true 43m
general (default) rbd.csi.ceph.com Delete Immediate true 68m
lvm-provisioner topolvm.io Delete WaitForFirstConsumer true 51m
Storage Classes¶
For each volume group with the lvm-csi function, the LVM CSI application
creates a storage class. This feature requires that all nodes have the same
number, name, and type of volume groups for LVM CSI purposes.
The storage classes are created with WaitForFirstConsumer volume binding
mode, which means the PVC will remain in Pending state until pod
assignment.
To list available storage classes, execute:
sysadmin@controller-0:~$ kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
cgts-vg topolvm.io Delete WaitForFirstConsumer true 94m
lvm-provisioner topolvm.io Delete WaitForFirstConsumer true 94m
Snapshot¶
The LVM CSI application supports the deployment of snapshot resources for
thin logical volumes only. Although a Volume Snapshot Class is created for each
storage class with the lvm-csi function, snapshot operations will fail for
thick-provisioned volumes. If your workload requires snapshots, ensure that the
target VG is configured with thin provisioning.
For more details on this limitation, see Restrictions and Limitations.
To list the available volume snapshot classes, execute:
~(keystone_admin)]$ kubectl get volumesnapshotclasses.snapshot.storage.k8s.io
NAME DRIVER DELETIONPOLICY AGE
cgts-vg topolvm.io Delete 3h18m
lvm-provisioner topolvm.io Delete 3h18m
To create a snapshot of a thin volume, you only need to deploy a snapshot resource as described below:
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: my-snapshot
spec:
volumeSnapshotClassName: lvm-provisioner # <---- Use the same name of the system
source:
persistentVolumeClaimName: my-pvc-src # <---- Use the same name of your pod's PVC
Scheduler¶
TopoLVM offers a dedicated scheduler to provide a better experience for multi-node clusters. The deployment of the scheduler is automatically managed based on the cluster configuration:
All-in-one simplex (AIO-SX): The Kubernetes standard scheduler is used. The scheduler pod is not expected to be running.
Other multi-node configurations: The TopoLVM dedicated scheduler is used. A scheduler pod is expected on each controller node.
This behavior can be changed by applying an override for the LVM CSI application:
cat <<'EOF' > overrides.yaml
scheduler:
enabled: false # <---- Or true
EOF
system helm-override-update lvm-csi lvm-csi topolvm-system --values overrides.yaml
The scheduler only analyzes node capacity, however in situations of multiple kinds of storage devices (HDD, NVME, etc) a prioritization method can be used.
For more details, see the official documentation topolvm-scheduler.
Remove Volume Groups¶
During application usage or at any other time, it is possible to remove a VG configured for LVM CSI purposes. However, removal is only allowed if the VG has no LVs in use.
The number of LVs in each VG can be found using the host-lvg-show or host-lvg-list command.
For CGTS-VG:
The number of LVs in use for LVM CSI purposes can be found in the
thin_cur_lv value within the parameters field.
~(keystone_admin)]$ system host-lvg-show controller-0 cgts-vg
+-----------------------+--------------------------------------+
| Property | Value |
+-----------------------+--------------------------------------+
| lvm_vg_name | cgts-vg |
| vg_state | provisioned |
| uuid | 850d583a-8490-4cb6-a4ea-bfc70af57b36 |
| ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 |
| lvm_vg_access | wz--n- |
| lvm_max_lv | 0 |
| lvm_cur_lv | 13 |
| lvm_max_pv | 0 |
| lvm_cur_pv | 1 |
| lvm_vg_size_gib | 261.375 |
| lvm_vg_avail_size_gib | 28.484 |
| lvm_vg_total_pe | 66912 |
| lvm_vg_free_pe | 7292 |
| lvm_function | lvm-csi |
| lvm_type | thin |
| lvm_pool_size | 42 |
| created_at | 2026-07-31T10:17:55.470587+00:00 |
| updated_at | 2026-07-31T16:15:23.827720+00:00 |
| parameters | {'thin_cur_lv': 0} |
+-----------------------+--------------------------------------+
For dedicated disks:
The number of LVs can be found in the “Current LVs” column.
~(keystone_admin)]$ system host-lvg-list controller-0
+--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+
| UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs |
+--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+
| 30a38897-bfb2-4c27-9e4c-1457e0a12f55 | lvm-provisioner | provisioned | lvm-csi | thick | None | wz--n- | 9.761 | 9.761 | 1 | 0 |
| 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 34 | wz--n- | 261.375 | 34.492 | 1 | 13 |
+--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+
Root Disk — Remove the lvm-csi Function from CGTS-VG¶
To remove the lvm-csi function from CGTS-VG, follow the steps below:
List the available VGs.
It is not possible to remove the
lvm-csifunction from a VG that is in use. Ensure that all PVCs and LVs provisioned by the LVM CSI application have been deleted before proceeding.~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+-----------------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+-----------------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 24 | wz--n- | 261.375 | 24.515 | 1 | 14 | +--------------------------------------+-----------------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Change the function by assigning
noneto the-fparameter.~(keystone_admin)]$ system host-lvg-modify controller-0 cgts-vg -f none +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | lvm_vg_name | cgts-vg | | vg_state | provisioned | | uuid | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | | ihost_uuid | 5aee8aa2-2b09-4492-afcd-dee80d65e278 | | lvm_vg_access | wz--n- | | lvm_max_lv | 0 | | lvm_cur_lv | 13 | | lvm_max_pv | 0 | | lvm_cur_pv | 1 | | lvm_vg_size_gib | 261.375 | | lvm_vg_avail_size_gib | 34.492 | | lvm_vg_total_pe | 66912 | | lvm_vg_free_pe | 8830 | | lvm_function | none | | lvm_type | None | | lvm_pool_size | None | | created_at | 2026-07-31T10:17:55.470587+00:00 | | updated_at | 2026-07-31T16:12:48.999759+00:00 | | parameters | {'thin_cur_lv': 0} | +-----------------------+--------------------------------------+
Verify the updated VG configuration with the host-lvg-list command.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+-----------------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+-----------------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | none | None | None | wz--n- | 261.375 | 48.562 | 1 | 14 | +--------------------------------------+-----------------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Dedicated Disks — Remove Volume Groups¶
Note
It is suggested to execute these commands with the host in online/unlocked state.
List the available VGs.
It is not possible to remove a VG with the
lvm-csifunction that is in use. Verify that the “Current LVs” column shows a zero value before proceeding.~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+ | 30a38897-bfb2-4c27-9e4c-1457e0a12f55 | lvm-provisioner | provisioned | lvm-csi | thick | None | wz--n- | 9.761 | 9.761 | 1 | 0 | | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 34 | wz--n- | 261.375 | 34.492 | 1 | 13 | +--------------------------------------+-----------------+-------------+----------+-------+-----------+--------+------------------+------------------+-------------+-------------+
Execute the deletion. This command does not produce output on success.
~(keystone_admin)]$ system host-lvg-delete controller-0 lvm-provisioner
List the VGs to confirm the deletion.
~(keystone_admin)]$ system host-lvg-list controller-0 +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | UUID | LVG Name | State | Function | Type | Pool Size | Access | Total Size (GiB) | Avail Size (GiB) | Current PVs | Current LVs | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+ | 850d583a-8490-4cb6-a4ea-bfc70af57b36 | cgts-vg | provisioned | lvm-csi | thin | 34 | wz--n- | 261.375 | 34.492 | 1 | 13 | +--------------------------------------+----------+-------------+----------+------+-----------+--------+------------------+------------------+-------------+-------------+
Remove the LVM CSI Application and LVM Storage Backend¶
Once configured, the LVM CSI application can only be removed using the
--force option. This safeguard prevents accidental removal.
The LVM CSI application is a managed application, which means that after removal, if the minimum configuration is still in place, the system will reapply the application automatically.
Removing the application does not affect user resources. All PVCs, PVs, pods, and system configurations remain intact. However, after removal, any administrative operation on pods, PVCs, or PVs that were provisioned by the LVM CSI application may lead to errors or data loss.
To remove the application:
Use the application-remove command with
--force.~(keystone_admin)]$ system application-remove lvm-csi --force +---------------+----------------------------------+ | Property | Value | +---------------+----------------------------------+ | active | False | | app_version | 26.10-0 | | created_at | 2026-07-31T11:43:14.949353+00:00 | | manifest_file | fluxcd-manifests | | manifest_name | lvm-csi-fluxcd-manifests | | name | lvm-csi | | progress | None | | status | removing | | updated_at | 2026-07-31T11:43:49.441479+00:00 | +---------------+----------------------------------+ Please use 'system application-list' or 'system application-show lvm-csi' to view the current progress.
Confirm that the application is now in
uploadedstatus.~(keystone_admin)]$ system application-list +--------------------------+-----------+-------------------------------------------+------------------+----------+-----------+ | application | version | manifest name | manifest file | status | progress | +--------------------------+-----------+-------------------------------------------+------------------+----------+-----------+ | lvm-csi | 26.10-0 | lvm-csi-fluxcd-manifests | fluxcd-manifests | uploaded | completed | +--------------------------+-----------+-------------------------------------------+------------------+----------+-----------+
To remove the storage backend:
~(keystone_admin)]$ system storage-backend-delete lvm-store --force ~(keystone_admin)]$
Backup and Restore Considerations¶
LVM CSI resources are included in backup and restore operations, but the following key concepts apply:
Only LVs provisioned on dedicated disks can be restored.
Any LV created on CGTS-VG (root disk) will be deleted during the restore process.
During the restore process, the user can decide whether LVs should be wiped or not.
Execute the Backup¶
No additional operation is needed to include LVM resources in the backup file.
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/backup.yml -v \
-e "ansible_become_pass=<PASSWORD>" \
-e "admin_password=<PASSWORD>" \
-e "backup_dir=/opt/platform-backup/localhost" \
-e "backup_registry_filesystem=true"
Execute the Restore¶
During the restore operation, the user can choose whether LVM resources
should be wiped by using the wipe_lvm_disks flag. This flag wipes all LVs
while preserving the VG structure from the backup, but without the data.
The default value for wipe_lvm_disks is false.
AIO-SX
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -v \
-e "ansible_become_pass=<PASSWORD>" \
-e "admin_password=<PASSWORD>" \
-e "initial_backup_dir=/opt/platform-backup/localhost" \
-e "backup_filename=$(ls /opt/platform-backup/localhost/*_platform_backup_*.tgz | tail -n -1 | xargs basename)" \
-e "restore_mode=optimized" \
-e "restore_registry_filesystem=true"\
-e "wipe_lvm_disks=true"
Other configurations
ansible-playbook /usr/share/ansible/stx-ansible/playbooks/restore_platform.yml -v \
-e "ansible_become_pass=<PASSWORD>" \
-e "admin_password=<PASSWORD>" \
-e "initial_backup_dir=/opt/platform-backup/localhost" \
-e "backup_filename=$(ls /opt/platform-backup/localhost/*_platform_backup_*.tgz | tail -n -1 | xargs basename)" \
-e "wipe_lvm_disks=true"
Restrictions and Limitations¶
This section presents the limitations and restrictions for the LVM CSI application on StarlingX. It is not intended to reproduce the official documentation. For additional information, consult the TopoLVM documentation: https://github.com/topolvm/topolvm
Root Disk Usage¶
Enabling CGTS-VG for LVM CSI purposes only supports thin provisioning. Any attempt to configure CGTS-VG with thick provisioning will be rejected.
Deletion of LVM CSI Volume Groups¶
Before any removal operation — especially VG removal — ensure that the
resource is not in use. VGs with the lvm-csi function cannot be deleted
if any pod, PV, or PVC depends on an LVM-provisioned resource.
For dedicated VGs, the number of LVs in use is shown in the “Current LVs” column of the system host-lvg-list <hostname> command.
For the root disk, the number of LVs in use for LVM CSI purposes can be
found in the thin_cur_lv parameter using the system host-lvg-show
<hostname> cgts-vg command.
Any attempt to remove a VG that is in use will be rejected by the system.
Change the Provisioning Type of an Existing Volume Group¶
Changing the provisioning type from thin to thick or from thick to thin is not supported. If it is necessary to change the provisioning type of an existing VG, the procedure is:
Remove the VG.
Wait for the system to update.
Create a new VG with the desired provisioning type.
Nodes with Different Volume Group Names or Availability¶
In multi-node configurations, the LVM CSI application requires that all nodes
have the same number of VGs and the same VG names with the lvm-csi
function enabled. It is possible to enable only one node to run LVM CSI
resources, but if more than one node is used, all participating nodes must have
the same configuration.
Example 1: Acceptable configuration
Controller-0 |
Controller-1 |
Compute-0 |
|---|---|---|
cgts-vg (with |
cgts-vg (with |
cgts-vg (with |
lvm-provisioner (dedicated thick VG) |
lvm-provisioner (dedicated thick VG) |
lvm-provisioner (dedicated thick VG) |
Example 2: Non-acceptable configuration
Controller-0 |
Controller-1 |
Compute-0 |
|---|---|---|
cgts-vg (with |
cgts-vg (without |
cgts-vg (with |
lvm-provisioner (dedicated thick VG) |
lvm-provisioner (dedicated thick VG) |
lvm-provisioner (dedicated thick VG) |
Example 3: Non-acceptable configuration
Controller-0 |
Controller-1 |
Compute-0 |
|---|---|---|
cgts-vg (with |
cgts-vg (without |
cgts-vg (with |
lvm-provisioner (dedicated thick VG) |
lvm-provisioner (dedicated thin VG) |
If one or more hosts have a non-compliant configuration, the application will fail to apply and a message will be shown in application-show. The previously applied configuration will not be affected.
~(keystone_admin)]$ system application-show lvm-csi
+---------------+------------------------------------------------------------------------+
| Property | Value |
+---------------+------------------------------------------------------------------------+
| active | True |
| app_version | 26.10-0 |
| created_at | 2026-07-31T12:57:00.271652+00:00 |
| manifest_file | fluxcd-manifests |
| manifest_name | lvm-csi-fluxcd-manifests |
| name | lvm-csi |
| progress | All hosts must have the same LVGs with lvm-csi function. Host eae559b6 |
| | -d72e-439c-87f5-184ec7f44540 has different LVGs than the others. |
| status | apply-failed |
| updated_at | 2026-07-31T12:58:10.555315+00:00 |
+---------------+------------------------------------------------------------------------+
Snapshots Can Only Be Created for Thin Volumes¶
The LVM CSI application supports snapshots only for thin volumes. This should be considered during cluster configuration.
Only Dedicated Volumes Can Be Restored¶
All LVs created using the CGTS-VG storage class will be deleted during the restore process. If data must be preserved during restore, it is recommended to copy the data to the platform-backup partition or to an external volume.
Partitions Cannot Be Restored¶
Using disk partitions as LVM CSI resources is allowed; however, data on partitions cannot be restored during the restore process. If data must be preserved during restore, it is recommended to copy the data to the platform-backup partition or to an external volume.
