1. Home
  2. Popular
Log in to post
  • olivierlambertO

    πŸ›°οΈ XO 6: dedicated thread for all your feedback!

    Pinned Xen Orchestra
    249
    7 Votes
    249 Posts
    101k Views
    poddingueP
    Depends which one you mean, because there are two different problems tangled together in this stretch of the thread and they have opposite answers. If you mean @escape222's report at #177, where a VM cloned from XO 6 sits at the TianoCore screen for a couple of minutes, then yes, that was an XO bug and it is fixed. PR #9867, merged 26 May, shipped in XO 6.5.0 on 28 May. The boot order was being rewritten whenever no new disk needed provisioning, so an HVM VM created from a template that already had a disk got network pushed to the front whether or not anyone asked for a network install, and the VM burned the PXE timeout before falling through to the disk. It now follows the install method only. Since you are on sources, anything past 6.5.0 has it. If you mean the slow UEFI boot @MajorP93 described at #182, with the installing Xen timer and spinlock lines, that one is not an XO bug and no XO patch will touch it. It is a regression in the Linux guest kernel introduced in 6.12.5. The cost lands per secondary vCPU, so the wider the VM, the worse it looks. There is a separate thread with the per-vCPU numbers: Ubuntu cloud images on XCP-ng 8.3 UEFI. Worth noting its title blames console=ttyS0, which we now think amplifies the same bug rather than being a second one. I measured that one here this week on a single host, changing only the guest kernel between runs and leaving everything else alone. Ubuntu 7.0.0-30 came in at 50 and 59 seconds across two runs. 7.0.0-31 came in at 0.4. Wall clock reboot to sshd went from 78 seconds to 31. The awkward part is the timing. The upstream fix is f24df84cbe05, in stable 6.12.97 and later, 6.18.y, 7.1.4 and later, and 7.2, but no default channel carries it yet. I re-checked the archives this evening: Debian trixie still ships 6.12.94-1, with 6.12.100-1 sitting in proposed-updates for the next point release, and Ubuntu 26.04 still ships 7.0.0-30 in updates, published today, while 7.0.0-31 has been in proposed since 10 August. Rocky and el10 I could not confirm either way. So keep whatever workaround you are on until a named version lands for your distro. There is arguably a third one at #183, where @Greg_E had Debian 13 and Windows Server 2022 refusing to boot at all when created through XO-lite with UEFI. As far as I know nobody has retested that since. If it is the kernel one you are hitting, this says which version you are waiting for: uname -r dmesg -T | grep -iE "installing Xen timer|spinlock event"
  • D

    Ubuntu cloud images on XCP-ng 8.3 UEFI: ~15s per secondary vCPU at boot, caused by console=ttyS0

    Unsolved Compute uefi cloud-init slow-boot ubuntu
    7
    0 Votes
    7 Posts
    110 Views
    poddingueP
    Update, and dvinni beat me to most of it: Canonical already has the fix in 7.0.0-31, and his SRU timing is better than anything I had. Two things I can still add from a lab box. First, 7.0.0-30 does not carry the patch, and it only landed in resolute-updates this morning, so an apt upgrade today leaves you where you were. Second, on keeping the ttyS0 removal in cloud-init templates, I was wrong to imply -31 makes that pointless. Same VM on 7.0.0-31, three boots each way, systemd-analyze: console=tty1 console=ttyS0 10.95 / 11.14 / 11.23 s console=tty1 7.40 / 7.38 / 7.47 s So the serial console is still worth about 3.7s of an 11s boot, spread across kernel, initrd and userspace rather than sitting in any one stage. That's a 6 vCPU VM on a little i5 lab box, so don't expect the same figure on your Xeons, but it's independent of the timekeeping bug, which is the part you had right and I talked past. I also need to withdraw the wall-clock numbers from the first version of this post. I was timing reboot to sshd, and when I ran the same config twice it gave me 7s and 31s, so it wasn't measuring boot. Probably the network coming up rather than the boot itself, though I didn't chase it. The sched_clock and systemd-analyze figures held across every run. Still wouldn't put a proposed kernel on a production pool on my say-so. Until it promotes, your ttyS0 change looks like the cheaper of the two workarounds.
  • stormiS

    XCP-ng 8.3 updates announcements and testing

    Pinned News
    653
    1 Votes
    653 Posts
    517k Views
    A
    @gduperrey Rolling pool update worked with released production patches.
  • msupportM

    Veeam 13.1 Rocky9 Linux Appliance: Potential Data Loss with CBT and Workers with Expired Tokens

    Unsolved Backup
    10
    2 Votes
    10 Posts
    497 Views
    acebmxerA
    @msupport - Looks like veeam is still working with your on your issues. While veeam has pushed me off to vates / xen. @poddingue - Any updates from Vates about these issues? Is it possible the least patches just pushed might help with either mine or @msupport's issue?
  • henri9813H

    Slow boot on rocky linux 10 latest kernel

    Unsolved Compute
    27
    2
    0 Votes
    27 Posts
    3k Views
    poddingueP
    Coming back to this with what the distros actually ship, because "the fix is upstream" turned out not to mean much on its own. The commit is f24df84cbe05. I checked each distro by grepping kernel/time/jiffies.c at the version they ship, rather than comparing version numbers, since some of them cherry-pick. Already fixed, nothing to do: Fedora 44: 7.1.8 Alpine 3.22: linux-virt 6.12.103. Alpine 3.23 and edge: 6.18.44 Not fixed in what you get today: Debian 13: trixie ships 6.12.94, which doesn't have it. 6.12.100 and 6.12.101 do, and they're in trixie-proposed-updates, so it should land with the next point release. Ubuntu 26.04: -updates moved to 7.0.0-30 this morning and that doesn't have it either. 7.0.0-31 does, sitting in resolute-proposed, expected early September. Rocky 10 I couldn't settle. I don't see it in the CentOS Stream 10 kernel changelog, which does list per-commit subjects, and Stream is at 6.12.0-260 while @henri9813 is on the 10.2 branch at -211. So probably not yet. But I might just be failing to find it, so if someone can check properly I'd rather be corrected. @acebmxer @MajorP93 on Debian, and anyone on Ubuntu: keep tsc_mode=2 and nomigrate for now. One thing that won't help you there, the console=ttyS0 removal from the other thread is an Ubuntu cloud image thing. Debian's cloud image recipe only sets a serial console for Azure, EC2 arm64 and ppc64el, so I don't think the generic amd64 image carries it. Worth a look at /proc/cmdline on yours though. I did check that the fix works rather than assuming it: same VM, Ubuntu 7.0.0-30 against 7.0.0-31, and the sched_clock correction went from about -59s to -0.4s on 6 vCPUs.
  • E

    After Update XO wont start

    Xen Orchestra
    6
    1
    1 Votes
    6 Posts
    684 Views
    D
    Hey that sucks after catching up 23 commits. Looks like something broke in the log module during the update. I’d try a clean reinstall of the dependencies first, or drop back to an older Node version for a bit – Node 24 can be fussy with these packages.
  • B

    Update Templates

    Xen Orchestra
    14
    1 Votes
    14 Posts
    3k Views
    D
    Hi bikemuch, For unregistered XOA the template auto-update is limited. Easiest way is via CLI: download the latest Debian 13 and CentOS Stream 10 ISOs, then use xe vm-import or create new templates from them. You can also check the XOA β€œTemplates” section and force a refresh if available. Works fine for me this way.
  • K

    Tesla xenctrlext.unix_error Device or resource is busy

    Unsolved Hardware
    13
    2
    0 Votes
    13 Posts
    2k Views
    poddingueP
    @seanmcg182 's post might be the answer to the original question, two years late. Your lspci -s 0000:05:00.0 -v at post 8 shows (rev ff) and !!! Unknown header type 7f, which is the same pair he had, and in his case it came from bifurcating the wrong PCIe port in the BIOS rather than from anything XCP-ng was doing. He says the unknown header is what then produced the device or resource busy error on VM start, which is the error you opened with. I don't know whether your board exposes bifurcation the way his X10DRH-CT does, so it might not transfer at all. @kuznetcoff777 if that machine is still around, it'd be worth a look either way.
  • D

    Smart Reboot blocked in XO, and no Rolling Pool Update

    Unsolved XCP-ng
    9
    0 Votes
    9 Posts
    277 Views
    D
    @poddingue said: What I can't tell you is what set that particular combination on your VM in the first place. Does it ring a bell? I have no Idea. I had it on "Protect from accidental shutdown" but turned that off again, later. Doing this again (on, off) helped, as you said. Thank you so much!