I deleted the snapshot from this one vm. [image: 1752005046814-ef2e8d88-d35e-4014-9db4-91032fbb9551-image.png] However, that one remains in the list on here: [image: 1752005088423-f469155b-9463-4b24-8d75-2dc4d1b72356-image.png]

@olivierlambert Dang ok. I waited a few minute then clicked the Connect in XOA for that host and it connected. Not sure what to do really.

Do you have any SR using a VM? (like an ISO SR in a NFS share inside a VM). This is freezing NFS and makes host taking half an hour to restart. Logs should tell you why the RPU failed if it failed

It's impossible to answer right off the bat without knowing more in details what's going on. HA is a complex beast, and combined with HCI, requires a lot of knowledge to find what's causing your issue, between both xha and XOSTOR. In other words, it is very demanding to analyze all the logs and trying to make sense of it. However, I can give you some clues to make sense of it: The HA log is at /var/log/xha.log. When you shutdown a host, you should be able to watch (on each host) what the HA is deciding to do. My gut feeling: there's maybe a XOSTOR issue making the heartbeat SR being unavailable, and so all hosts will autofence Then you need to understand the XOSTOR logs for why the cluster wasn't doing what's expected. My best advice: remove HA first, and only then investigate on XOSTOR. Kill on node (not the master) and check if your VMs are still able to start/snapshot/write inside.

@dnikola said in [HELP] XCP-ng 4.17.5 dom0 kernel panic — page fault in TCP stack, crashdump attached: Has anyone experienced similar page faults in the dom0 TCP stack on 4.19 kernels or XCP-ng 4.17.5? Not that I know of. Are there any known issues with network drivers on this kernel/hypervisor combo? No, there can be issues with some drivers, you should have specified which network NICs and drivers you are using. Would you recommend moving to a newer dom0 kernel or hypervisor build? On XCP-ng, the latest version is 8.3 which you didn't specify in your post, but you're using the latest version of Xen, so I assume it is an up to date 8.3, so there is no newer build. Could a memory issue cause this specific kind of page table inconsistency during a kernel panic? Yes, it can be a bug in the the code, but it absolutely could be a hardware issues. Any advice on additional debug steps or log files I should collect next time? I would start by running a memtest on that host to make sure the memory is not having issues. Do you know if there was a specific VM doing something specific at that time? We had some issues in the past with FreeBSD VMs using wireguard, but it does not look similar, and it should be fixed now. What kind of guests were running on that host? linux, windows, some BSD based? If running windows guests please be sure to have read this blog post and ensure to comply with the guidelines there. From a quick look, I don't see anything obvious. Follow Olivier's suggestion first, if you still have issues after that, you can share an additional report using xen-bugtool -y. But please be sure to update your bios first, check your memory, and then do that.

@JBlessing as it looks like it does start, it looks like the networking side is working, at least at first. Just for debugging purpose you could try to switch that VM to BIOS instead of UEFI if it is possible, maybe it is related to what the pxe is starting in the VM. You could also try switching the VM between realtek and e1000 NIC, at this stage, PV drivers are not there so it is using an emulated NIC, maybe the image your PXE starts doesn't like the one you're using and it gets stuck somehow. As you're already using it with vmware, I assume you know how to size your VM, but if you went for a tight RAM value for this VM, you could try to give it more RAM to see if that could be related, as everything has to fit in RAM at some point, we may be using more at startup than vmware… Hope one of this can help

Will have to think more on this. We do have a a domian policy for Windows Updates, that mostly works; it never seems to work as intended. In any case, if a Citrix driver is updated, with the registry setting set, it will reboot x number of times? (3 is suggested I believe) Wish there was a bomb proof method. Just this morning I came to office to find half the building down, because MS decided to push a Citrix update last night, and the DHCP server was down, until I rebooted the VM from the console once more. ;(

About the xsconsole: sometimes it's not refreshing. You can try to get access to the console, then type "xsconsole" it will start it and you should see it works You must have the master up to date if you want your slave to connect again. I never tried to elect a new master in the middle of the upgrade, I would discourage it. Better shutdown some VMs on the master, upgrade it and you are automatically back on track.

Then try to find anything happening around that time on other hosts, equipment, storage and so on.

Hello @msupport we published a step by step guide, read more in the announcement there https://xcp-ng.org/forum/post/94268

Great, thank you!

@Danp yes I ran "yum update" to be sure but "nothing to upgrade" on the pool-master. I try with storage (iscsi) NIC configured and without Storage NIC configured but the pool join freeze. Seems that persist some "SESSION" (may be referred to the slave host previously configured?) or some incoerence in the pool database... from /var/log/xensource of the slave host when try to join the pool: "session_check D:520c5b4e5b36 failed with exception Server_error(SESSION_INVALID, "

Thanks, this is helpful We'll discuss that with the @Team-DevOps and try to get things implemented!

To me it sounds like a Commvault issue. If you want some investigation on Vates side, I would recommend to open a support ticket

@olivierlambert Agreed. The Citrix forum used to be very active, but especially since Citrix was taken over, https://community.citrix.com has had way less activity, sadly. It's still gratifying that a lot of the functionality still is common to both platforms, although as XCP-ng evolves, there will be continually less commonality.

@olivierlambert During RPU - yes. i mean manual update in case of failure.

For anything older than the branches still shown in https://pkgs.alpinelinux.org, (from v3.0 to v3.12), the packages should be downloaded from their cdn: https://dl-cdn.alpinelinux.org/alpine But as mentioned above, anything older than 3 releases from the lastest current one(v3.21) are end of life and should not be used for more than testing.

@snowstruk The stock XCP 8.3 r8125 driver works for my 8125 and 8126 devices. I have not tried it a 5G, so it's possible the 8126 is not working at 5G (I don't see it advertised) with the current driver. I have run the current dedicated realtek 8125 driver and it works but I have not tried the 8126 driver. I don't know how many people are using the 8126 chipset (I guess not many). If your device is not working at >1G then it sees to be a hardware incompatibility issue. You could try loading the new 8126 firmware to see if that helps. Here's the output about my test 8126 card (attached to a 2.5G switch): # lspci -v -s 04:00.0 04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8126 5GbE Controller (rev 01) Subsystem: Realtek Semiconductor Co., Ltd. Device 0123 Flags: bus master, fast devsel, latency 0, IRQ 18 I/O ports at 3000 [size=256] Memory at e0d00000 (64-bit, non-prefetchable) [size=64K] Memory at e0d10000 (64-bit, non-prefetchable) [size=16K] Expansion ROM at e0d20000 [disabled] [size=64K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [70] Express Endpoint, MSI 01 Capabilities: [b0] MSI-X: Enable+ Count=32 Masked- Capabilities: [d0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [148] Virtual Channel Capabilities: [170] Device Serial Number 01-00-00-00-b4-b3-c9-88 Capabilities: [180] #19 Capabilities: [190] Transaction Processing Hints Capabilities: [21c] Latency Tolerance Reporting Capabilities: [224] L1 PM Substates Capabilities: [234] Vendor Specific Information: ID=0002 Rev=4 Len=100 <?> Kernel driver in use: r8125 Kernel modules: r8125 # ethtool -i eth1 driver: r8125 version: 9.012.04-NAPI-PTP-RSS firmware-version: rtl8126a-2_0.0.2 02/01/24 expansion-rom-version: bus-info: 0000:04:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: yes supports-priv-flags: no # ethtool eth1 Settings for eth1: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseX/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseX/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Link partner advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseX/Full Link partner advertised pause frame use: Symmetric Receive-only Link partner advertised auto-negotiation: Yes Link partner advertised FEC modes: Not reported Speed: 2500Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on MDI-X: Unknown Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000033 (51) drv probe ifdown ifup Link detected: yes

Reping @stormi

I already suggested you the solution, now it's up to you to live with those process or to decide to reboot (ideally after doing updates because it's very dangerous to NOT being up to date)