@mietek I can assure you there's no malicious intent to mislead you, why we would do that to our users is beyond me. Olivier is simply one of the most busy people I've worked with, and he still takes the time to come here and answer free users when he can. He might not have the time to scour the internet and github for the relevant patches and news for niche threads like this. We were not made aware of the cloud-init fix until just a few days ago when a patch was submitted. Cloud-init has been notoriously hard to support and document because the upstream project is constantly doing things like this, and as you noticed it can affect how it works on one OS version versus another very differently. If it were up to me we would drop built-in support for it because of this mess (and a lot of large projects have dropped it entirely and moved to Ignition, like CoreOS) but a lot of users still find it very useful so we continue to support it as best we can, baring with the mess going on upstream.

I'm not sure if you're aware, but Olivier is the founder and CEO of Vates, who is behind both XCP-ng and XOA. I welcome you to go to the ESXI forum and try and get the CEO of VMware to personally answer your questions, as a free user to top it off.

Why is it these people can never spell?

@alexanderk Heh yes, and a few other Brocade/Quanta/Dell guides

+1 for avoiding 3rd party install scripts, the official install doc is like, 10 commands? I think it takes maybe 10 minutes or less the last time I did it on a fresh debian system, and you know it's always the correct instructions (which is NOT the case with 3rd party scripts, as you'll see in this forum when even the slightest architectural change is made to the XO sources). You also get to learn at least a little bit about the architecture of what you're installing and running instead of pressing go on a script you grabbed from some guy's github and hoping the XO web interface appears. I thought the whole point of "homelab" (for which sources are intended and primarily used) was learning and developing skills in the first place?

No disrespect intended to the people that create and maintain said scripts, it just seems to me like it bypasses the point of sources a little bit

@maxcerny Indeed, the Ubuntu cloud-init template should be up by the end of this week (cloud-init has so many fun bugs to work around for these applications)

I can start adding to this this weekend probably - would a "guide to installing pfsense" be useful? I know there's many guides out there already, but more than half of them have useless (or worse than useless) steps telling people to turn off things that don't need to be turned off

@s-pam If you don't mind about power usage, high performance (that you've already selected) is the way to go. It will avoid entering C states as often and leave the CPU clocked high among other things so there is no latency for workloads waiting on the CPU existing C states etc.

Leave x2APIC enabled, it helps to distribute interrupts between multiple cores/CPUs. There's no downside to having it enabled unless the hypervisor doesn't support it (and you would know immediately by having errors).

Regarding your last setting, also referred to as CCX as NUMA, this changes how the CPU cores are presented to the hypervisor (either as one NUMDA domain, or multiple, one for each set of cores that share cache). From the benchmarks I can find, leaving this on gains maybe 3 to 4 percent improvement https://blogs.vmware.com/performance/2020/05/app-perf-vmware-vsphere-amd-epyc-rome.html

@olivierlambert indeed, I believe it's just:

xe host-management-reconfigure pif-uuid=<PIF_UUID>

with the PIF_UUID being the UUID of the new PIF interface (bond, vlan, etc) you would like to move the existing management interface to

@station-already Hi! This can be done in XOA, but it requires deleting the existing management interface, then creating a new one on the new PIF you've created (a vlan 10 on eth0 PIF for example). You say you can't lose access to the host however, so that is probably not an option. XCP-ng center, the windows client, has a bit of a workaround, it has a dropdown box that will allow you to change the PIF of an existing management interface/IP address to a new interface. In XCP-ng center just go to the Networking tab of the host, then hit configure at the bottom beneath the management interfaces section.

It will have a new pop up window with a dropdown for where you want the existing management interface to exist. Assuming you've already created the vlan network you want in xcp-ng, it should show up in this list. just choose it and hit ok:

@olivierlambert I believe he means be able to rename / map a hosts physical interfaces to different namings. For instance on host 1, "network 0 / eth0" would be mapped to the physical eth0 interface, but on another host in the pool with a different NIC configuration, he could map the same pool "network 0 / eth0" to the server's physical eth1.

@S-Pam this certainly isn't currently supported, I believe it would take a lot of rewriting the network backend to make it possible.

@alexanderk Heh yes, and a few other Brocade/Quanta/Dell guides

@maxcerny I'm not sure on a debian 11 image, if it's been assigned to another team member it may be done very soon, if not I can probably get one created within the next week.

As for bugs, take your pick. For a "vanilla" install it's not too bad, but a big can of worms gets opened when you add in an avenue to pass network information to the cloud-init instance - it either overriding your cloud-init.cfg and ignoring it, or not overwriting interfaces and just appending to it leaving a stale DHCP address and lease, not overriding resolvers that were set previously during the templating process, stuff like that. Then there's the fun stuff like /etc/cloud/cloud.cfg.d/99-installer.cfg and other files in this directory overriding the nocloud config even when explicitly commented out. Many of these things have been reported to the cloud-init project but it seems everyone has moved to Ignition (which I'd like to do in the future, but it would take a lot of rewriting on the XOA side). This GIST (not mine) sums up some of my feelings pretty well - https://gist.github.com/Informatic/0b6b24374b54d09c77b9d25595cdbd47

@alexanderk Hi, apologies but we were never able to trace this down to anything in the xen kernel itself, it seems to be something specific to the older gen 8 HP and how their BIOS/UEFI firmware handles VT-d / passthrough. If there's newer BIOS firmware available for your machine, you could try that, and double check vt-d, IOMMU etc is enabled in bios.

@maxcerny Indeed, the Ubuntu cloud-init template should be up by the end of this week (cloud-init has so many fun bugs to work around for these applications)

@olivierlambert I think the request makes sense, it would be really helpful to see at a quick glance what type of bond it is and what are the bond members (especially for support)

@zevgeny Hi, yes this can be easily done by just creating a new network as you described. VMs will be able to access anything in that network/subnet without extra configuration. If they ever need to talk to things outside of that network/subnet (like the internet), they would need a router to route the traffic between subnets

@cbaguzman hi, you could try a hard mount instead of soft, then it should constantly retry until it sees the mount again https://www.ibm.com/docs/en/spectrum-protect/8.1.6?topic=considerations-nfs-hard-soft-mounts

@olivierlambert like I said in my previous reply I believe this would take quite a bit of rewriting in XAPI to accommodate pools with mismatched hardware. The interface rename tool might work in the meantime? https://xcp-ng.org/docs/networking.html#renaming-nics

@mike hmm, NFS locking is handled by statd so make sure that is running properly on your NFS server.

Also, this may be related to the .nfsXXXX files that are generated/used by the system to keep track of open files that should later be deleted when closed. These things:

root@zfs[/mnt/chunker/xen-backup/XH-01]# ls -lh
total 18
-rw-r--r--   1 xen  wheel     0B Oct 19 05:24 .nfs000000000000020300000002
drwx------   3 xen  wheel     3B Jun 29 05:59 xo-config-backups
drwx------   3 xen  wheel     3B Jun 29 05:59 xo-pool-metadata-backups
drwxr-xr-x  15 xen  wheel    15B Sep 26 09:39 xo-vm-backups

In your backup of the backup job, can you have it exclude any .nfs* files so it doesn't try to access or copy them, and see if the locking errors go away?

@dr4s1l-0 @Dr4s1l-0 you will not see the config drive mounted in your system, cloud-init only mounts it temporarily very early on in the boot process, copies the found config files off of it, and unmounts it. Mounting it manually will just cause more issues. You can probably see it being mounted early on in the logs in /var/log/cloud-init.log

Cloud-init has introduced a ton of bugs this year and has no documentation regarding them, it has been a nightmare to try and track down these issues - I have a support customer with the same issue on recent versions of ubuntu and I've gotten nowhere. At this point I think it's best to take it up with the upstream project, cloud-init - from the XOA/XCP perspective we're doing everything we can, resulting in presenting a properly configured config drive - any issue after that is the cloud-init package