First SMAPIv3 driver is available in preview

[hoerup]

From the blog post

Size: grow a VDI in size. It's fully thin provisioned!

Does this mean we finally have live/hot disk expansion ?

[olivierlambert]

Not right now, but likely in the future, yes

[ddbolt]

@olivierlambert said in First SMAPIv3 driver is available in preview:

right now, but likely in the future,

Are we going to eventually be able to have larger than 2TB volumes on EXT4 partitions? I'm not really setup to do ZFS and not really sure I want the headache that come along with it. My PERC controllers have always been super reliable so Ext4 is fin with me as for FS trype.

[olivierlambert]

We will likely not use VHD anymore in SMAPIv3 (there's no point), so yes, if/when we decide to make an ext4 driver, it will store other files format (like maybe qcow2)

[Tristis Oris]

created, clone some VMs into, run, looks fine.
Hard to say anything about perfomance, etc. Need a some real production (impossible), or compare benchmarks (fio for example, but ZFS should be slower of basic lvm anyway, so what result is good?). Is it anything we need to test?

provisioning unknown)

same for VM, probably > no stat gathering with SMAPIv3.

[nikade]

Yeah its not going to display any stats for now, so you'll have to look at the numbers inside the VM.

[olivierlambert]

The goal is to test the fact that it runs OK for a bit, so we are sure to not miss anything. Fio is your friend to benchmark in a VM, remember that it's still blktap behind, so if you want better performance numbers, do it with multiple VDIs at once.

[gskger]

I tested SMAPIv1 on XCP 8.2.1 against SMAPIv3 on XCP 8.3b2 using the same host (a HP ProDesk 400 G6 with a i5-10500T CPU, 32GB RAM). A 1 TB Samsung 860 EVO SSD drive was used as the test SR, while XCP was booted from a 512 M.2 KIOXA NVMe drive. Fio (fio-3.37) was compiled from source on an up-to-date Debian 12 VM (2 vCPU, 4 GiB RAM, 32GiB drive) which was copied twice so that three identical VM could run fio in parallel.

After an initial fio run to create the files, a script run three sequential write and read tests (e.g. fio --name=fio --ioengine=libaio --randrepeat=1 --direct=1 --fallocate=none --ramp_time=10 --size=4G --iodepth=64 --loops=50 --group_reporting --numjobs=1 --rw=write --bs=1M). The script first ran on one VM, followed by a run on three VMs in parallel. IOPs and bandwidths were averaged.

v1-1VM are the results for one VM on a SMAPIv1 SR (XCP 8.2.1) while v3-3VM are the results for three VMs in parallel an a SMAPIv3 SR (XCP 8.3b2).

While I'm not sure if this approach is really valid (e.g. the average load of the host went through the roof when three VMs performed fio in parallel), it does suggest that the bandwidth of SMAPIv3 is not yet en-par to that of SMAPIv1. But I could be wrong and this is an early previews of SMAPIv3. Looking forward to more performance results on SMAPIv3.

[olivierlambert]

Hi,

I'm not sure to understand. What kind of SMAPIv1 SR did you try to compare with ZFS on v3?

[rfx77]

@olivierlambert

Can you provide a link to the github repo where we can find the source-code of this smapiv3 driver?

[olivierlambert]

https://github.com/xcp-ng/xcp-ng-xapi-storage

[rfx77]

@olivierlambert
i meant the source for this package: xcp-ng-xapi-storage-volume-zfsvol

so that we can see how this new driver is implemented

[olivierlambert]

That's inside the repo I posted

[CJ]

Has anyone tried a backup using the new driver? I created a new test pool with one of my previous hosts and made SMAPIv3 ZFS storage. I can create a VM just fine, but when I try and add it to my existing backup job, it keeps erroring out with "stream has ended with not enough data (actual: 485, expected: 512)"

Is this expected?

[olivierlambert]

You can only do full backup for now, not incremental.

[CJ]

@olivierlambert Since it's the first backup, it should be full, correct? Does Delta backup not work at all even if force full is enabled?

[olivierlambert]

I mean the backup feature, it only works with XVA underneath (so the full backup feature that is doing a full everytime)

[hsnyder]

I've started using the SMAPIv3 driver too. It's working well so far. I'm keeping my VM boot disks on md raid1, and using a zfs mirror via SMAPIv3 for large data disks.

I have a question about backups... Is it safe to use syncoid to directly synchronize the ZFS volumes to an external backup? syncoid creates a snapshot at the start of the send process. But, I also have rolling snapshots configured through Xen-Orchestra. Will the syncoid snapshot mess up Xen-Orchestra?

If this isn't safe or isn't a good idea, I'll just use rsync to back up the filesystem contents inside the VM that the volume is mounted to...

[olivierlambert]

On my side, I have no idea, because I never used syncoid. Have you asked their dev about this?

[rfx77]

@olivierlambert @hsnyder

if i understand correctly i would rephrase the question this way:

does xen-orchestra name the snapshots in a way which is unique to xen-orchestra and does xoa know which snapshots belong to it or does it use the latest snapshots no matter how they are named.

@hsnyder: i dont think you can simply use zfs snapshots without xen snapshots because it dont think that they will be crash-consistent.

if syncoid is similar to zrepl you have to check that is doesnt prune the zfs snapshots from xoa.