Support VMware-to-KVM Migrations for BFV-Only Instances via OpenStack Resize by anokfireball · Pull Request #627 · sapcc/nova

anokfireball · 2026-06-12T08:36:44Z

Implements the VMware-to-KVM Cross-Hypervisor Instance Migration for boot-from-volume instances (only!) based on the ADR decision by adding (1) the conductor sanitization of image properties that would pin the instance to VMware HVs, (2) the required source-side VMware driver control flow, (3) the confirm/revert control flow that commits/puts those properties back, and (4) an operator-visible rollout gate so activation is decoupled from deployment.

The cross-HV branches are guarded by _is_cross_hv_resize which delegates to a shared is_supported_cross_hypervisor_resize helper. Currently the only allowlisted transition is VMware vCenter Server -> CH. The API and conductor both use raise_on_unsupported_cross_hypervisor_resize to enforce preconditions (BFV-only, instance must be RUNNING) and reject unsupported directions at the request API level.

The commits map to the ADR's Resize Orchestration choice:

compute: add cross-HV image property journal. Sanitizes img_hv_type and the VMware bus/vif/scsi/video model fields from request_spec.image.properties before _schedule(), so the scheduler can pick a KVM/CH host. Originals go into a new MigrationContext.old_image_properties field (versioned object v1.2.1) for revert. _move_claim carries the journal across the destination claim. Revert restoration lives here too: request_spec.image.properties and system_metadata['image_*'] are restored from the journal before _update_volume_attachments and the driver call so the FCD driver receives VMware-era connection semantics again.
compute: Allow VMware-to-CloudHypervisor resize. Adds the shared detection helper (is_supported_cross_hypervisor_resize) and the API-level precondition gate (raise_on_unsupported_cross_hypervisor_resize). The conductor's MigrationTask._prep_cross_hv_resize sets cross_hv_resize and cross_hv_source_prepared markers in system_metadata before dispatching to the VMware driver.
vmware: enable cross-HV BFV resize in VMware driver. Adds branches to migrate_disk_and_power_off, confirm_migration, and finish_revert_migration. Source-side power-off and volume/NIC detach use ReconfigVM only: no file deletion, no Cinder calls. The "shell" VM stays registered in vCenter under migration.uuid so confirm and revert can find it. Source preparation is only marked complete after the shell is fully created, and confirm/revert skip shell operations when the source was never prepared. Revert restores instanceUuid before re-attaching volumes, since the volume attach paths look up the VM by instance.uuid.
compute: gate cross-HV resize behind feature flag. Adds [workarounds] enable_cross_hv_resize (default False) and rejects cross-HV resize at the API/conductor validation boundary until operators explicitly enable it after rollout. Together with the v1.2.1 OVO patch bump, this lets us ship the code without activating the feature until every service in the resize path has been rolled out.
compute: translate device names for cross-HV resize. VMware reports volume BDM device names as sd* while KVM/CH presents virtio disks as vd*. When cross_hv_resize is set, BDM device_name values and instance.root_device_name are translated from sd* to vd* in _finish_resize before _update_volume_attachments, and back from vd* to sd* in _finish_revert_resize on the same boundary. Cinder mountpoints stay synchronized with no extra Cinder API calls.

Also as per ADR, there is no Cinder retype during resize. The FCD driver keeps serving NFS connection_info to KVM connectors, and volumes stay typed vstorageobject. Customers run a standard os-retype when they are ready to change the volume type.

BzzyBriel · 2026-06-16T15:56:17Z

        compute_utils.heal_reqspec_is_bfv(
            self.context, self.request_spec, self.instance)
+        self._old_image_properties = {}
+        src_hv = self._source_cn.hypervisor_type if self._source_cn else None


self._source_cn can not only be None in tests but also from this (obsolete) codepath. Should I go fix the tests but leave the check in or just leave the check in without fixing the tests?

grandchild · 2026-06-26T12:04:29Z

i've squashed & reordered the commits a bit

Problem Cross-HV resize sanitizes VMware image properties before scheduling, but revert needs the original values in both the RequestSpec and instance system_metadata. Solution Add the RequestSpec sanitizer and MigrationContext.old_image_properties journal. Preserve that journal when MoveClaim rebuilds the migration context, then use it to restore RequestSpec image properties and system_metadata image_* keys during revert. Introduce the field as MigrationContext VERSION 1.2.1 to avoid colliding with a future upstream 1.3. Change-Id: I56271a118ec3785eb5ba81a6f648e58d18169e73

Allow VMware vCenter Server to CH resize for running BFV instances only. Add shared cross-hypervisor resize detection and API/conductor guardrails so unsupported directions are rejected before scheduling. The conductor sanitizes image properties and writes cross-HV system_metadata intent markers before dispatching resize work. Co-authored-by: Jakob Karge <jakob.karge@sap.com> Co-authored-by: Fabian Koller <github@kthxbye.cyou> Change-Id: I50dedca220e70f51811c0f9b9327ca3ae7ea9e12

Problem The VMware driver needs source-side shell handling for cross-HV BFV resize. The source VM must stay available as a recoverable shell so confirm and revert can complete safely. Solution Add cross-HV branches to migrate_disk_and_power_off, confirm_migration, and finish_revert_migration. The source VM is powered off, volumes and NICs are detached, instanceUuid is swapped to migration.uuid, and the shell is retained for confirm or revert. Use the shared supported-transition helper for detection, mark source preparation only after shell creation completes, and keep rollback UUID-aware when the VM has already been renamed to migration.uuid. Change-Id: I3db51a7dc0cb1dfc1592959d9823250a93fd3358

Problem Cross-HV resize depends on every service in the resize path understanding the new MigrationContext journal and driver state. During rolling upgrades, activating the feature before all services are updated can cause RPC object backport timeouts. Solution Add [workarounds] enable_cross_hv_resize, defaulting to False, and reject cross-HV resize at the API/conductor validation boundary until operators explicitly enable it after rollout. Change-Id: I2ec2513dbad1caeea7e6d99368bd4789f1ecb388

Problem VMware reports volume BDM device names as sd*, while KVM/CH presents virtio disks as vd*. After cross-HV resize, BDMs and instance.root_device_name otherwise keep the VMware-era names, and Cinder receives stale mountpoints during attachment update. Solution When cross_hv_resize is set, translate BDM device_name values and instance.root_device_name from sd* to vd* in _finish_resize before _update_volume_attachments. Translate back from vd* to sd* in _finish_revert_resize before the same attachment update path. This keeps Cinder mountpoints synchronized without extra Cinder calls. Change-Id: I66742bfac43541cdafe8f43bde751394

grandchild · 2026-07-02T13:46:18Z

+CROSS_HV_SANITIZE_PROPS = (
+    ('img_hv_type', None),
+    ('hw_disk_bus', 'virtio'),
+    ('hw_cdrom_bus', 'virtio'),
+    ('hw_scsi_model', None),
+    ('hw_vif_model', 'virtio'),
+    ('hw_video_model', 'virtio'),
+    ('img_hv_requested_version', None),
+)


making this a dict will prevent the (admittedly unlikely) error of listing a field twice, thus overwriting the true old value in the old return value of sanitize_image_props_for_kvm().

BzzyBriel · 2026-07-02T13:29:43Z

+                restore = (
+                    compute_utils.restore_image_props_from_cross_hv_journal)
+                changed = restore(sysmeta=instance.system_metadata,
+                                  old_image_properties=(
+                                      mig_ctx.old_image_properties))


Suggested change

restore = (

compute_utils.restore_image_props_from_cross_hv_journal)

changed = restore(sysmeta=instance.system_metadata,

old_image_properties=(

mig_ctx.old_image_properties))

changed = \

compute_utils.restore_image_props_from_cross_hv_journal(

sysmeta=instance.system_metadata,

old_image_properties=mig_ctx.old_image_properties)

I think it's clearer to call the function directly instead of assigning it to a variable to reduce line length.

anokfireball changed the title ~~Cross hv vmware driver~~ Add VMware-to-KVM BFV resize Jun 12, 2026

anokfireball mentioned this pull request Jun 12, 2026

compute: RequestSpec sanitization for cross-HV resize #623

Closed

anokfireball changed the title ~~Add VMware-to-KVM BFV resize~~ Add VMware-to-KVM BFV-only resize Jun 12, 2026

anokfireball force-pushed the cross-hv-vmware-driver branch from 7de8abf to fa741a2 Compare June 16, 2026 09:11

BzzyBriel force-pushed the cross-hv-vmware-driver branch 3 times, most recently from bb2df8b to bd4f689 Compare June 16, 2026 15:51

BzzyBriel reviewed Jun 16, 2026

View reviewed changes

BzzyBriel force-pushed the cross-hv-vmware-driver branch from bd4f689 to 074afbd Compare June 16, 2026 15:58

anokfireball commented Jun 17, 2026

View reviewed changes

Comment thread nova/conductor/tasks/migrate.py Outdated

anokfireball commented Jun 17, 2026

View reviewed changes

Comment thread nova/exception.py

anokfireball commented Jun 17, 2026

View reviewed changes

Comment thread nova/conductor/manager.py Outdated

BzzyBriel reviewed Jun 18, 2026

View reviewed changes

Comment thread nova/compute/api.py Outdated

anokfireball commented Jun 18, 2026

View reviewed changes

Comment thread nova/compute/utils.py

grandchild force-pushed the cross-hv-vmware-driver branch from 8dc1531 to 5ce32e2 Compare June 26, 2026 07:32

grandchild marked this pull request as ready for review June 26, 2026 07:36

grandchild marked this pull request as draft June 26, 2026 07:36

grandchild force-pushed the cross-hv-vmware-driver branch from 5ce32e2 to 3d4db3a Compare June 26, 2026 12:03

grandchild force-pushed the cross-hv-vmware-driver branch from 3d4db3a to 4fb113c Compare June 26, 2026 12:27

anokfireball commented Jun 29, 2026

View reviewed changes

Comment thread nova/virt/vmwareapi/vmops.py Outdated

anokfireball commented Jun 29, 2026

View reviewed changes

Comment thread nova/objects/migration_context.py

anokfireball commented Jun 29, 2026

View reviewed changes

Comment thread nova/virt/vmwareapi/vmops.py Outdated

anokfireball force-pushed the cross-hv-vmware-driver branch from e681d24 to 2c92e16 Compare June 30, 2026 08:12

anokfireball marked this pull request as ready for review June 30, 2026 09:55

anokfireball requested a review from a team June 30, 2026 12:13

fwiesel reviewed Jun 30, 2026

View reviewed changes

Comment thread nova/objects/migration_context.py

anokfireball changed the title ~~Add VMware-to-KVM BFV-only resize~~ Support VMware-to-KVM Migrations for BFV-only instances via OpenStack Resize Jul 1, 2026

anokfireball changed the title ~~Support VMware-to-KVM Migrations for BFV-only instances via OpenStack Resize~~ Support VMware-to-KVM Migrations for BFV-Only Instances via OpenStack Resize Jul 1, 2026

BzzyBriel and others added 2 commits July 2, 2026 09:59

anokfireball force-pushed the cross-hv-vmware-driver branch from 2c92e16 to e444104 Compare July 2, 2026 08:07

anokfireball added 2 commits July 2, 2026 10:46

anokfireball force-pushed the cross-hv-vmware-driver branch from e444104 to 1713ae5 Compare July 2, 2026 09:10

fwiesel approved these changes Jul 2, 2026

View reviewed changes

Comment thread nova/objects/migration_context.py

grandchild reviewed Jul 2, 2026

View reviewed changes

BzzyBriel approved these changes Jul 2, 2026

View reviewed changes

grandchild approved these changes Jul 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support VMware-to-KVM Migrations for BFV-Only Instances via OpenStack Resize#627

Support VMware-to-KVM Migrations for BFV-Only Instances via OpenStack Resize#627
anokfireball wants to merge 5 commits into
stable/2023.2-m3from
cross-hv-vmware-driver

anokfireball commented Jun 12, 2026 •

edited

Loading

Uh oh!

BzzyBriel Jun 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

grandchild commented Jun 26, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

grandchild Jul 2, 2026

Uh oh!

BzzyBriel Jul 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

anokfireball commented Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BzzyBriel Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

grandchild commented Jun 26, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

grandchild Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

BzzyBriel Jul 2, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

anokfireball commented Jun 12, 2026 •

edited

Loading

BzzyBriel Jun 16, 2026 •

edited

Loading