dthenot

dthenot

Hello,

I'm a new developer on XCP-ng, I'll work on the Xen side to improve performance.
I'm a newly graduated of University of Versailles Saint-Quentin with a specialty in parallel computing and HPC, I have a big interest in operating systems.

dthenot

Hello,

As some of you may know, there is currently a problem with disks with blocksize of 4KiB not being compatible to be a SR disk.
It is an error with the vhd-util utilities that is not easily fixed.
As such, we quickly developed a SMAPI driver using losetup ability to emulate another sector size to be able to workaround the problem for the moment.

The real solution will involve SMAPIv3, which the first driver is available to test: https://xcp-ng.org/blog/2024/04/19/first-smapiv3-driver-is-available-in-preview/

To go back to the LargeBlock driver, it is available in 8.3 in sm 3.0.12-12.2.

To set it up, it is as simple as creating a EXT SR with xe CLI but with type=largeblock.

xe sr-create host-uuid=<host UUID> type=largeblock name-label="LargeBlock SR" device-config:device=/dev/nvme0n1

It does not support using multiple devices because of quirks with LVM and the EXT SR driver.

It automatically creates a loop device with a sector size of 512b on top of the 4KiB device and then creates a EXT SR on top of this emulated device.

This driver is a workaround, we have automated tests but they can't catch all things.
If you have any feedbacks or problems, don't hesitate to share here

dthenot

@ph7 It's only enabled for the two yum command with the --enablerepo explicitly used.
It's disabled in the config otherwise.
No need to do anything

dthenot

@gb.123 Hello,
The instruction in the first post are still the way to go

dthenot

@nasheayahu Hello,

I do this in my homelab with a disk since I imported it from a physical machine:

mkdir /srv/NAS
xe sr-create type=udev device-config:location=/srv/NAS name-label="NAS Disks" host-uuid=<UUID of the host with the disks>

Then you make a symlink to the device:

ln -s /dev/sda /srv/NAS/sda #although it might be better to use a stable identifier if you have multiple disks
xe sr-scan uuid=<UUID of the udev SR>

The disk will appear as a VDI in the SR that you can then plug to a VM.

I also renamed the VDI [NOSNAP][NOBAK] NAS Drive
[NOSNAP][NOBAK] instruct XO to not snapshot and export the disk in backups (since it can't).

It's not passthrough, as in the disk is not directly given to the VM, but tapdisk (the process virtualizing storage) will give access to the whole disk to the guest, so it can mount any FS on it.

The udevSR is usually used for plugging USB from what I could gather from existing code. But users in homelabs have been using it like this for a while.
Though you will likely not have native performance level since it's not exactly passthrough.

dthenot

@Andrew Hello,

I have been able to find the problem and make a fix, it's in the process of being packaged.
I can confirm it only happen for file based SR when using purge snapshots.
For some reason, the vdi type of CBT_metadata is cbtlog for FileSR but stays the image format it was for LVMSR
And it would make a condition fail during the list_changed_blocks call.

dthenot

@manilx Hi,

yum install plug-late-sr

Should do the trick to install it

dthenot

@olivierlambert In 8.2 yes, linstor sm version is separated, it's not the case in 8.3 anymore.

dthenot

@ccooke Hello,

We have a fix, we are aiming to validate it rapidly so it shouldn't happen for others.
Thank you for reporting the issue. I'll update the thread again when the update is available and it should be safe for other people going through here to update using the RPU.

dthenot

@olivierlambert I am

dthenot

@samuelolavo Hello,

Could the fact that this is a local LVM SR on a non-master host be relevant to how make_chain_rw is being handled?

Yes, it's indeed what's happening. The check for the SRMaster (for a shared SR it's the master but for a local it's supposed to be done on the host directly) is failing and always saying False. I'll fix this.

Thank you for the help

dthenot

@samuelolavo Hello,

It is indeed linked to QCOW2.

Is mixing VHD and QCOW2 VDIs on the same VM and on the same SR fully supported?

Yes, it's supposed to work.

Are there specific minimum versions of xapi, sm, blktap, or other storage components required for large QCOW2 VDIs?

Yes, but we would prefer for people to be uptodate, we have been fixing things.
But I don't think it's a problem for you at the moment.

make_chain_rw is necessary for the master to make the QCOW2 LV RW at the VDI activation time. So it's likely linked to this particular QCOW2.

make_chain_rw is executed on the master host and is logging in /var/log/SMlog, could you take a look at what it's erroring on?

dthenot

@Dan Hello,

So our investigation found that this change was introduced with commit https://github.com/xcp-ng/sm/commit/00637dd52e845d6016add9718fc9cc694aec9f0d
It was aimed at EXTSR in particular, it appear that the first SR to be plugged is the one choosing the mode of sr-mount since util.makedirs also create the parent directory with the given mode.

I have created a card for the issue on our side.

dthenot

@Dan Hello,

I can't answer from memory, we'll take a look at this and come back to you

Thanks for the report, I guess it could be considered a bug since even it's one of the other, it's not the expected behaviour to have the mode change between runs.

dthenot

@bvitnik It's specific to XCP-ng.

dthenot

@bvitnik Hello,

sm-config:vdi_type is the pre-existing one so we are still writing it for compatibility reasons.
But you might notice that vdi_type is only written for LVM-based SRs.
While image-format is always written in XCP-ng with the QCOW2 feature.

So it might be safe to just look at image-format if it exist and if it's not, it's likely a VHD.
On LVMSR, you can fallback on vdi_type (and it should work for XS too).

Indeed, ISO SRs do not advertise a VDI format. Technically, ISOs could be considered to be raw.

It should also be noted that our implementation of QCOW2 is very different from GFS2 QCOW2 of XenServer.
Our implementation uses the existing SR types to allow VHD and QCOW2 to co-exist.

dthenot

@probain Hello,

It's likely linked to the List index out of range bug.
That bug was linked to the SR scan failing to introduce CBT_metatadata VDI in the XAPI database, could you try to launch a xe sr-scan uuid=<SR UUID> and try again to disable CBT?
If it does not work, could you share the /var/log/SMlog of around the time you are trying to disable CBT?

dthenot

@Andrew Hello,

I'm also getting error on some VMs while trying to export a disk and also trying to even start some VMs from NFS (that were fine before).

Is it still a problem? It might have multiple causes. If it's still an issue, could you share the logs: /var/log/{SMlog,xensource.log,daemon.log}. It contains information that could help us investigate.

The VDI not detached cleanly means that the VDI still has a reference to the host it was running on before.
It's might be caused by the xenopsd error earlier or maybe it's the cause.

If you are sure no tapdisk are using the VDI, you can look at tap-ctl list output on each host.
You can clean this reference with the script in /opt/xensource/sm/resetvdis.py single <VDI UUID>.

dthenot

@nasheayahu Sorry, I made a mistake, it's device-config:location --'
I'll edit my first post with the correct configuration, you will also likely need to give the host-uuid of the host the disk is on.

dthenot