XCP-ng Team

Developers of XCP-ng

Posts

  • RE: Migrating an offline VM disk between two local SRs is slow

    I'm not sure disabling Nagle is a good idea (even though it can improve things here). Fundamentally, we're doing bulk transfer of disk content, which Nagles tries to optimize by coalescing packets, so you're not flooding the network with small TCP packets.

    The main problem here is that the progress is gated by NBD replies, which is going to be bad regardless of TCP configuration. TCP_NODELAY will workaround this problem, but with significant tradeoffs (and perhaps will perform worse in some other cases).
    What should be done instead is that writes should be streamed (or pipelined) while reading replies in parralel, so that NBD reply delays doesn't bottleneck the whole transfer. But that actually requires a redesign of the whole NBD implementation which is not going to be a easy thing AFAICT.

  • RE: Migrating an offline VM disk between two local SRs is slow

    Worth mentioning @Team-Storage

  • RE: XCP-NG 9.0 Support for GRAID Tech GPU-accelerated RAID cards.

    I can't speak for the whole team, but will still give my view on it, at least on what I'm aware of.

    @hydromike a dit:

    ...

    VROC is the easier half

    Graid licensed the Intel VROC IP outright — the deal closed last November, and they relaunched it as "VROC by Graid Technology" at Computex on 1 June with a 24-month roadmap and Xeon 6 support. Channel rollout started this quarter.

    VROC on Linux is Intel VMD plus the kernel's own md driver with IMSM metadata. Their 2.0.1 release notes confirm it — graidctl ls vd -t VROC returns /dev/md123. In-tree code. No proprietary module, no kernel taint, no NVIDIA driver in dom0.

    Roughly what that path needs:

    • CONFIG_VMD in the dom0 kernel
    • The VMD-under-Xen work @yann already did once for the demonstrator (Xen support for VMD also landed upstream in 6.12.30)
    • mdadm with IMSM in dom0
    • Licensing that works without a hardware key — their roadmap replaces the physical key with UEFI-based licensing, which would help

    Worth reminding that installing custom software in Dom0, or replacing Dom0 packages is discouraged.

    The part worth Vates' attention regardless of my use case: VROC RAID1 would give XCP-ng a supported dom0 boot mirror. That's been asked about here for years without an official answer. This is a plausible route to one.

    SupremeRAID is harder

    I don't want to undersell this. dom0 is a PV domain, not bare metal — memory-capped, limited vCPUs, doesn't own the IOMMU, and its physical addresses aren't machine addresses. If the SupremeRAID data path depends on peer-to-peer PCIe DMA between the GPU and the NVMe drives, that either works or it doesn't, and no amount of packaging effort fixes it. Their install guide also recommends disabling VT-d, which we obviously can't do. Those questions have gone to Graid and I'll post what comes back.

    P2P DMA is more of a platform capability (e.g ACS stuff) than a hypervisor issue. At least, as long as there is no DMA to magic regions like MSI-X/PBA as that will potentially fail.

    In the past, I had success doing P2P DMA between a PCI card and NVMe drives, so it's definetely doable.

    ...

    Questions for @yann and @Team-Hypervisor-Kernel

    These decide whether Graid's engineers should spend any time at all, so I'd rather ask now than after:

    1. Is dom0 in 9.0 still PV, or has it moved to PVH? This changes the DMA and IOMMU picture substantially and it's the first thing their team will ask.

    Dom0 will still be running in PV mode by default at least. There are no immediate plan to switch to PVH Dom0.

    1. What's the target kernel for 9.0, and roughly when does the ABI freeze? I don't want to point them at 6.12 if the foundations rebuild lands elsewhere.
    2. What's Vates' position on a proprietary, kernel-tainting out-of-tree module in dom0 — acceptable in principle, or a hard no? DRBD and now the backported Ceph client are both already in dom0, but both are GPL, so I read those as precedent for the mechanism rather than the licensing.
    3. Is a supplemental pack a viable delivery vehicle, or would this need to go through Koji as a normal -module RPM?

    Features like SecureBoot will add complexity with external out-of-tree modules, and while there can be solutions to that, but I don't expect any of that to land anytime soon.

    1. Is CONFIG_VMD enabled in the 9.0 kernel builds, and did the VMD patch make it in?

    If question 3 is a no, that's a useful answer — it closes the SupremeRAID track cleanly and we put the effort into VROC instead.

    ...

    AIA, I don't expect anything practical with the current state of things unless there is a quite significant effort.
    However, if these stack(s) can be moved in a dedicated VM with PCI Passthrough, that can make things more workable with the likes of storage domains.

  • RE: XCP-ng 8.3 updates announcements and testing

    @manilx If the pools are up to date, I would try restarting the XO VM to see if the task persists.

    If it does persist, it might be best to open a separate topic for the XO team.

  • RE: XCP-ng 8.3 updates announcements and testing

    Thank you everyone for your tests and your feedback!

    The updates are live now: https://xcp-ng.org/blog/2026/08/27/august-2026-updates-2-for-xcp-ng-8-3-lts/

  • RE: XCP-ng 8.3 updates announcements and testing

    New maintenance update candidates for XCP-ng 8.3 LTS

    This batch of updates focuses on storage bug fixes and performance improvements.

    What changed

    Storage

    • blktap:
      When CBT is enabled, an issue occurred where tapdisk opened cbtlog instead of the primary QCOW2 image. This previously caused commit operations (ENOTSUPP) and leaf-coalesce routines to fail, resulting in excessive storage consumption on the SR. Tapdisk now correctly bypasses the cbtlog image to allow proper coalescing.

    • sm: Fix a bug where the coalesce on a secondary host of a QCOW2 VDI on LVMSR could corrupt LVM metadata and would need manual intervention to restore them.

    • xapi:
      - Migration speeds are now improved for sparse QCOW2-backed VDIs, as empty sectors are no longer transferred.
      - Preserve VDI tags on migration. This is a foundational fix for kubernetes integration.

    Versions

    • blktap: 3.55.5-9.3.xcpng8.3 -> 3.55.5-9.4.xcpng8.3
    • sm: 3.2.12-23.4.xcpng8.3 -> 3.2.12-23.5.xcpng8.3
    • xapi: 26.1.16-1.1.xcpng8.3 -> 26.1.16-1.2.xcpng8.3

    Test on XCP-ng 8.3

    yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
    yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
    reboot
    

    The usual update rules apply: pool coordinator first, etc.

    What to test

    As usual, normal use and anything else you want to test.

    Test window before official release of the updates

    ~24 hrs

    We would like to thank users who shared feedback since our last call for testing: @Andrew, @JeffBerntsen, @MajorP93, @XCP-ng-JustGreat, @acebmxer, @bufanda, @flakpyro

  • RE: Intermittent Xen blkfront I/O stalls: all guest tags busy while tapdisk reports zero outstanding requests

    @mike.potapov Can you upgrade to the latest blktap-3.55.5-9.3.xcpng8.3 to check is the issue is still there?

Member List

olivierlambertO olivierlambert Group Owner
22,473 Posts 3,389 Reputation
nraynaudN nraynaud Group Owner
136 Posts 30 Reputation
stormiS stormi Group Owner
3,231 Posts 1,244 Reputation
ronan-aR ronan-a Group Owner
257 Posts 99 Reputation
BenjiReisB BenjiReis Group Owner
395 Posts 102 Reputation
gduperreyG gduperrey Group Owner
189 Posts 195 Reputation
LucienLassalleL LucienLassalle Group Owner
9 Posts 9 Reputation
asollierA asollier Group Owner
0 Posts 0 Reputation
ArnaudGFA ArnaudGF Group Owner
0 Posts 0 Reputation
M MrQ Group Owner
3 Posts 2 Reputation
erwan.crozeE erwan.croze Group Owner
0 Posts 0 Reputation
fred-stoF fred-sto Group Owner
1 Posts 0 Reputation
jul1anJ jul1an Group Owner
1 Posts 1 Reputation
vxgmichelV vxgmichel Group Owner
0 Posts 0 Reputation
C contificate Group Owner
0 Posts 0 Reputation
S sgerag Group Owner
0 Posts 0 Reputation
semarieS semarie Group Owner
36 Posts 8 Reputation
rzrR rzr Group Owner
73 Posts 49 Reputation
Emmanuel VE Emmanuel V Group Owner
2 Posts 2 Reputation
M Mathieu L Group Owner
4 Posts 2 Reputation