Installed on 3xDell PowerEdge R350 in HA. And so far, I haven't noticed anything out of the ordinary.

@MathieuRA

I noticed you merged https://github.com/vatesfr/xen-orchestra/pull/9787

I just tried it. And it does seem to fix my original issue!

Thank you! I am always impressed by you guys. Making testing and reporting upstream (to you guys) a good experience!

I submitted this as a GitHub issue last week.

TL;DR: Backblaze aparently doesn't support those flags that are enabled by default
"Backblaze does not yet accept these headers, so we recommend downgrading to AWS Javascript 3.x SDK version 3.728.0."

https://github.com/vatesfr/xen-orchestra/issues/8358

@poddingue
Our HA-pool is a 3 host system, yes.

I'll add a note about our plans to go paid in that feedback-item. Thanks for the tip.
In the meantime, I'm testing and reporting as much as I can. In order to hopefully help the product be better for all.

Cheers!

@poddingue

Huh.. Good catch.
I set all of the VMs to restart, and now it does make maintenance mode possible, on the host.

However, when we go into licensed/paid production (soon tm). It would be an absolute nightmare to try and find one VM that might not be on the right HA-plan... And it does kind of feel that Maintenance-mode should be compatible with whatever HA-plan is chosen regardless.

Perhaps there could be a way to assign a global/pool-wide HA-plan along with enabling HA alltogether?

Update: Shining light on that there is already a feedback-item, wishing for this kind of mass-operation for HA.
https://feedback.vates.tech/posts/45/bulk-operation-for-setting-vm-ha-option

@mpiton

Thanks for a fantasticly quick answer, and detailed as well. I will test this immediately!

Update:
I can confirm that this was indeed the problem!

@olivierlambert

Thanks again for your input and recomendations! I'll verify that this is solved by having the LUN expanded to 8GB instead. Afterwards I'll mark your answer as the solution!

@olivierlambert
I've finally had time to sit down and do some more troubleshooting.
It seems that the issue is somehow tied to the backup-jobs themselves.
Deploying XOA and subsequently importing XO-config from my XOCE instance. I continued to see several issues between both instances.
These issues pretty much went away, when I re-did the backup-jobs from scratch. And are now much more in-line with what I'm excepting to see. This was done on both XOA Stable, Latest and XOCE

I am no longer able to repoduce this at all.
So my guess is that a bug of some sort got introduced with this being a constantly updated from source instance.

Marking as solved. As I don't believe there is much more to do at this time.

@poddingue

Thanks for taking a look on your end as well.
I would like to point out that the line from the docs is quite deffinitive. If there are gotchas, such as UEFI affecting things. Then this either should be documented. Or better yet, that outlier of an edge-case probably should be addressed.

One scenario I'm envisioning is that a disk is faulty for some reason, and this goes unoticed due to the VM being rarely used. And since no backup-errors are given, this might go unoticed for quite some time. The Health Check passing is a false signal in that case.

These are my two-cents. And I confirm that this is an edge-case, and even unlikely one. But a theoretical greater than zero risk anyways.

Thank you

@McHenry
Differences could be things such as:

• Amount of cores between hosts for migrations
• Guarantee performance between guests. So that they don't affect each other as much.

Same with overprovisioning dynamic RAM

@poddingue

Thanks for taking a look.

Yes it is this part that tells me that it should start the VM automatically once the import has completed.
"After the transfer, the VM on XCP-ng side is started:"
Followed by, "This process is fully automated, without any human intervention after it starts on step 1."

https://docs.xcp-ng.org/installation/migrate-to-xcp-ng/

But my findings/experience is that the VM does not start once the import/transfer has completed.

So either the docs are wrong. Or the function is not behaving as intended. I'm acctually fine either way. As long as I know what to expect. However, I would like to have it auto-start the VMs after transfer completion. Which would make the procedure "Fully automated".

Update:
Am I assuming correctly that "the XO V2V guide says to start the VM yourself once the migration is complete " you're reffering to, is part of the "test migration"-procedure? Not the acctual production mgiration?
Because:

"Final migration
Run the production migration
When you're ready for the final migration:

Shut down the source VM completely.
Start the V2V migration in Xen Orchestra with the Stop source option enabled.
This ensures the final sync happens while the VM is powered off, and prevents any inconsistencies."

In my mind doesn't state that the VM should be started manually post-transfer.

Great thanks in advance for discussing!
Cheers!

I'm at the moment doing alot of test with V2V (applause on getting rid of the vddk-dependency btw )

However, what I'm noticing is that the migrated VMs don't automatically start when the import has completed. This is contrary to current documentation.

Is this a potential bug? Or am I missing something obvious?

PS: The imports are all sucessfull. It's just the auto-start part that I'm noticing doesn't work.

Cheers!

Commit: dc573

Template used for testig is: Rocky Linux 9

Third update:
It looks more and more that I had goofed something up in my testing. So please disregard everything in the post below.
The driver does work to make our LPe37000/LPe38000 series FC-cards work. Which is nice. Thanks!

~~Good morning

So I might've found an issue with this driver release. That I'm trying to troubleshoot and collect data on.

We have two LUNs presented to our dev-pool (xcp-dev), running on R650xs. When I have attached one LUN to xcp-dev I can't see the other LUN.
This works on our older R350s

Removing the attached LUN, makes a LUN visible again. But now we're getting some strange behaviour. And looking into if it is misconfigured from the Dell Powerstore, or if it is on the host(s) itself.~~

Details:
multipath is enabled

Host on xcp-dev

98:00.0 Fibre Channel [0c04]: Emulex Corporation LPe37000/LPe38000 Series 32Gb/64Gb Fibre Channel Adapter [10df:f500] (rev 11)
        Subsystem: Emulex Corporation Device [10df:f522]
        Kernel driver in use: lpfc
        Kernel modules: lpfc14
98:00.1 Fibre Channel [0c04]: Emulex Corporation LPe37000/LPe38000 Series 32Gb/64Gb Fibre Channel Adapter [10df:f500] (rev 11)
        Subsystem: Emulex Corporation Device [10df:f522]
        Kernel driver in use: lpfc
        Kernel modules: lpfc14

Update:
~~After managing to revert my previous tests and forget the existing (wrong) LUN from xcp-dev. I can now confirm a problem.

There are three LUNs. Let's call them 101, 102, 103

When I attach one LUN (103) to xcp-dev, I am no longer able to see the other LUNs. That means I can't add a second one. I am expecting to be able to see 102 available.

However, on old-dev, that has two LUNs attached already (101 & 102). It can see 103 as expected.~~

host on old-dev

02:00.0 Fibre Channel [0c04]: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100] (rev 03)
        Subsystem: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100]
        Kernel driver in use: lpfc
        Kernel modules: lpfc
02:00.1 Fibre Channel [0c04]: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100] (rev 03)
        Subsystem: Emulex Corporation LPe12000 Series 8Gb Fibre Channel Adapter [10df:f100]
        Kernel driver in use: lpfc
        Kernel modules: lpfc

Second Update:
This might be unrelated. Seems that we're having weird behaviour with the LUNs in general. Please disregard the statement above for now

@florent & @poddingue

I have now uploaded two heap-snapshots for you to look at.
If there is anything else, big or small, that I can help out with. Just let me know

Thanks!

I just did a quick test to check. And it does indeed seem that this (small) issue has now been resolved.
Tested on Windows Server 25, Management agent 9.2.385-0.

Cheers!

@poddingue said:

On the tuning suggestion, my guess, untested and purely from reading your paste, is that if something is genuinely holding sockets then raising tcp_mem buys headroom rather than stopping the accumulation, and your post#4 about throughput coming back on an xo_server restart points the same way.

It might be worth a mention to @Team-XO-Backend, since they would know whether xo-server keeps sockets per pool.

This is why I'm not increasing the numbers quite yet. Since this would only mask a potential issue. If an underlying cause is responsible, then this would be of interest.

Thanks for chiming in!

@poddingue

I have done some more testing.
Windows Server 2022 is also affected. 9.1.145-77

I also tried the latest WindowsPV drivers from Vates, on Windows Server 2025. 9.1.200-0. And the "problem" still persists.

It is a very low impact and low importance "problem". Since it is afaik purely costmetical.

@julienXOvates said:

@colomb said:

I really miss the description column XO5 has in the default/home view. I understand space is limited for the tree view, but maybe it could appear when a host is selected in the tree view?

In XO6, you will see it if you click on the eye at the end of the row, then on the right panel.
Is this OK ?

Personally I don't really find the rightmost pane/section all that usefull. It's to narrow to have any clear information. And it also leads to the other panels being smaller than they (imho) need to be able to show their base information as well.
This is especially noticable when using windows sizes narrower than 1080p.

The rightmost pane I hope will get either removed, and redone in UI v7. Because right now XO5 is far more informational than that is.

Sorry for jumping on with my two cents.

@MathieuRA
The sdn-plugin is back. I am amazed by that quick resonse and fix. THANKS!

However, the issue with @xen-orchestra/web/typed-router.d.ts being locally modified during compilation, is still present. Leading to have to git restore @xen-orchestra/web/typed-router.d.ts before being able to pull from github.

The build-script that I use is based on (then) official docs

#!/bin/bash
set -ex
set -o pipefail
git checkout .
git pull --ff-only
yarn
yarn build

Small discrepancy: VDIS or VDIs? Which one is it?

Red points out where it's called VDIS.
Green where it is called VDIs.

And as a bonus feedback/question. Why is it called "A VDIS" ?(marked with yellow)