@poddingue Turns out it is all secondary-to-secondary out of sync. So that makes it less intense, though if primary dies I wonder how it will resolve this, or if it will become split brained. Still unsure of how it happened, but the ones I manually cleaned up have not come back. Going to continue to manually clear them up. If it happens again I will have an alert setup to notify me, and I have all the xcp-ng logs and everything to be able to see what happened. If that happens I will post here with details and logs for the resource so we can see how it occurs. [image: image.jpeg]

Thanks for coming back with that. Commit 8c2f3 is PR #10518 (https://github.com/vatesfr/xen-orchestra/pull/10518), and the changelog lists it under XO 6.9.1 from 5 October, so XOA users get it with 6.9.1, on the latest channel for now if I read the release channels right.

Great, thanks for the feedback @yomeyo !

@poddingue Thanks for taking a look. Yes it is this part that tells me that it should start the VM automatically once the import has completed. "After the transfer, the VM on XCP-ng side is started:" Followed by, "This process is fully automated, without any human intervention after it starts on step 1." [image: image.jpeg] https://docs.xcp-ng.org/installation/migrate-to-xcp-ng/ But my findings/experience is that the VM does not start once the import/transfer has completed. So either the docs are wrong. Or the function is not behaving as intended. I'm acctually fine either way. As long as I know what to expect. However, I would like to have it auto-start the VMs after transfer completion. Which would make the procedure "Fully automated". Update: Am I assuming correctly that "the XO V2V guide says to start the VM yourself once the migration is complete " you're reffering to, is part of the "test migration"-procedure? Not the acctual production mgiration? Because: "Final migration Run the production migration When you're ready for the final migration: Shut down the source VM completely. Start the V2V migration in Xen Orchestra with the Stop source option enabled. This ensures the final sync happens while the VM is powered off, and prevents any inconsistencies." In my mind doesn't state that the VM should be started manually post-transfer. Great thanks in advance for discussing! Cheers!

If a mount uses the hard mount option, it will lock any process touching it, including unmount during shutdown, if the mount source is unavailable. It is a kernel thing and I believe there is no solution for it at present.

I didn't know, so I went and looked at the code. XO prunes that list in a cleanup step: backup runs are kept for 31 days, and other tasks are kept by count, the newest 1000 (https://github.com/vatesfr/xen-orchestra/blob/b59c8f3d20954aa1d72e92c1e8f2db635d74fcba/@xen-orchestra/mixins/Tasks.mjs#L119-L127). You can change both with tasks.gc.keep and tasks.gc.backupKeepDuration in the xo-server config. As far as I can tell the cleanup runs when xo-server starts, not on a timer, so a long-running XO can collect quite a few before the next restart. My guess for the Cloud Backup entries @acebmxer still sees after 30 days is that they're kept by count rather than by age.

@poddingue Our HA-pool is a 3 host system, yes. I'll add a note about our plans to go paid in that feedback-item. Thanks for the tip. In the meantime, I'm testing and reporting as much as I can. In order to hopefully help the product be better for all. Cheers!

@olivierlambert said: Hi, It's nice to suggest something, but we can't communicate just today about it, but be assured that we have stuff in the pipes to answer exactly all of this Hello Olivier, Understood completely on the timing—I appreciate that communications like this need to be perfectly aligned with the development cycle. It’s incredibly reassuring to hear that there is already a solution in the pipes! As someone who has been around the community since 2021, I’m more than happy to help keep the forum threads constructive and patient in the meantime. When the time comes, if you need any early testing, feedback, or GitHub contributions to help vet whatever you have planned, please feel free to loop me in. I’d be glad to help out. Looking forward to the announcement! Best, John

@pierrebrunet Tested and working again, thanks

@john.c @poddingue I am very happy to report that I am no longer receiving the "XOA Unable to connect xo server every 30s" message on the Chrome browser of my Windows 11 PC, as of today, after updating the XO-CE VM and all xcp-ng hosts. I just updated the XO-CE VM to master, commit 70b08, and ran a rolling pool update of my 3 xcp-ng hosts. Thank you to the Vates team.

@mpiton Thanks for a fantasticly quick answer, and detailed as well. I will test this immediately! Update: I can confirm that this was indeed the problem!

I tried it on a lab host (XCP-ng 8.3, xapi 26.1.16): calling VM.pool_migrate on a halted VM gives me your error, VM_BAD_POWER_STATE with running, halted, so the two values are the state XAPI wanted and the one it found. On the XO side, a migration inside the same pool with no SR or network mapping goes straight to VM.pool_migrate without checking the power state first: https://github.com/vatesfr/xen-orchestra/blob/b59c8f3d20954aa1d72e92c1e8f2db635d74fcba/packages/xo-server/src/xapi/index.mjs#L942-L981. So a stopped VM in that batch would give this, which fits what you remember. I haven't tried the batch migrate from the UI itself though, so I don't know if the UI is supposed to filter halted VMs out before that. It's a good concrete example for the feedback post anyway, XO could just say "this VM is halted" instead of passing XAPI's raw code through.

@ronan-a For testing purposes, I did the following: I created a VM on the local SR of a host, installed Linux on it, and configured an ISCSI target. I then added that ISCSI target as an ISCSI SR within xcp-ng pool, and created a small VM on that repository, using XO. While the machine was running, I performed a migration without losing a single ping.I assume this confirms my previous assertion regarding the use of a standard storage system.

Thanks, @Jonathon !

I wasn't even aware about this being a Windows limitation by itself

@samuelolavo Hello, Could the fact that this is a local LVM SR on a non-master host be relevant to how make_chain_rw is being handled? Yes, it's indeed what's happening. The check for the SRMaster (for a shared SR it's the master but for a local it's supposed to be done on the host directly) is failing and always saying False. I'll fix this. Thank you for the help

thanks @poddingue I've opened: https://github.com/vatesfr/packer-plugin-xenserver/issues/212

Hello, We have received your request and thank you for the feedback on this matter. A task card has been created and assigned to one of our teams. It has not yet been assigned to a specific developer, but we will be sure to get back to you once it has been addressed and resolved.

@anthoineb, Thanks for the fix! We plan to install blktap-3.55.5-11.1.xcpng8.3 on all three hypervisors tomorrow. We will fully shut down and start the data VMs one at a time, then verify that all running tapdisk processes use the updated binary. As the stalls are intermittent, we will monitor for recurrence and report back with the results.

@trelane Hello, the bug affects Xen drivers of any 9.0 series (considering the upstream code, it goes back to circa September 2019). 9.2.385 contains the fix, which was merged upstream just this month. The bug also impacts network connections, however to a lesser extent due to its nature. I believe that the bug is also present in XenServer VM Tools 9.6.0 and by extension, older versions of it. We recommend that you switch to the XCP-ng tools whenever possible, also due to its other benefits (fully open source, validated for XCP-ng and optimized for Xen Orchestra).