@mnv said in XCP-NG 9, Dom0 considerations:

@olivierlambert said in XCP-NG 9, Dom0 considerations:

@mnv For local SR, just use Local ext SR type and you will be thin already.

Hi Olivier,

Sorry for my late reply.

Yes, I am aware of that, however using LVM (like using ZVOL) would offer the disk as a block device directly, thin ext4 ought to be ext4 + the virtual disk + the VM filesystem, ZVOL/LVM should be block device + VM filesystem. There seems to be way less overhead.

ZVOL doesn't play nice with all SSDs, not mines for sure, LVM Thin seems to play much better, write less and take longer to have the same wear as ext4-thin or ZVOL.

I understand this isn't a problem in an Enterprise environment, though...

Larger issue is the existing block storage code is thick only, so rolls across to local lvm, HBA and iscsi. And fiber channel/SAS multipath HBA and iscsi are what are more commonly used in many enterprises, blocking many from migrating from competing products.
LVM has supported thin provisioning since release 2.02.89 back in 2012, with redhat 6.4 I believe.
This should also solve the initial vhd sparse file growth write performance penalty

@mnv said in XCP-NG 9, Dom0 considerations:

Thanks for your heads up, Olivier, and I hope your internal tests and developements go in the best direction! I can't wait for the next big thing! Only thing I could hope for that is missing (AFAIK) in XCP-NG 8.x is support for LVM Thin storage, that's so good when you have crappy consumer SSDs...

I've actually been puttering with LVM thin patches for awhile as it's always bothered me that we didn't have. I think I finally have my head around the code. , but I'm sure I'm missing something obvious that will be pointed out when I submit.
It's been a very-very-very-side project.

@steff22 said in Gpu passthrough on Asrock rack B650D4U3-2L2Q will not work:

@ravenet Ok I think the pro gpu's work better without so many bugs.

I switched to Nvidia RTX 4070. With the Nvidia gpu and this works as it should and gets an image on the screen on the first try.

So there must be something wrong with the amd drivers plus that error with the amd reset bug

I've setup a test system with a ryzen 7700x and a regular radeon 7600xt, non-pro gpu. Will let you know results. Shouldn't be an issue, it's same hardware and even the drivers work across them. Though they do have different bios on cards

@gduperrey
Tested on 4 systems in production

Ryzen 1700x, on asrock rack mb w radeon pro GPU pass through
Threadripper 5975wx on asrock rack mb w radeon pro GPU pass through
Epyc 9224 on Asus
Epyc 7313P on Asus

@gduperrey

Tested on Multiple systems. Ryzen 1700x andThreadripper 5975. fine so far

@steff22

I have a Radeon Pro W7500 working fine on Win11 vm, on xcp 8.3. Windows update even installed the pro series driver for me.
Asrock rack x470d4u, Ryzen 1700x.
I know it's older platform and agesa though. I may have access to a ryzen 7700x system for some testing next week.

@steff22
Far as I know, SVM, IOMMU, SR-IOV and POSSIBLY resize bar were enabled.
Can't check the threadripper system until the weekend as it runs production loads.

I'll reboot one of the Ryzen boxes tonight to verify what I enabled there, but was basically defaults other than making sure the 3 or 4 items above were enabled.

I have gpu pcie passthrough working on Asrock rack wrx80d8-2t (Threadripper), and x470d4u2 (Ryzen) to passthrough multiple Radeon Pro 7000 series gpus

Rodney

@stormi
Updated ryzen 1700 late last week, and Threadripper 5975 and Epyc 7313p servers. No issues so far.
Installing fresh on epyc 9224 this week

@Andrew Yes 8.3 will be xen 4.17. They've stated that this will be moved out of lab into the 8.3 official branch once they are happy with their and our feedback.

@Andrew said in XCP-ng 8.3 beta :

@ravenet I did not have any problems with my Mellanox Connect-4 Lx card during upgrades from 8.2.1 (5.0 driver), to 8.3 (5.4 driver), then update to current 8.3 (5.9 driver), and then update to Xen 4.17.3. My card has the 14.32.1010 firmware. It is not used for the management interface. It is used by VMs and continued to keep the same ethernet name and function for the VMs.

Thanks for testing

I finally got around to updating the firmware on this Mellanox connectx-4 LX adapter. Bit of jumping through hoops to install tools, find proper firmware, but went from FW version14.18.1000 to New FW version: 14.32.1010. Had to do an emergency network reset again, but it's back.

Obviously an incompatibility with the new 5.9 driver with much older mellanox firmware.

@olivierlambert said in XCP-ng 8.3 beta :

Thanks for your feedback @ravenet !

ill do firther testing later, in case issue is out-of-date firmware on the adapter itself being incompatible with the new driver. ill try and check the mellanox 5.9 changelog for firmware version requirements.

Testing now on ryzen 1700.

This update broke my mellanox network drivers for connectx-4 / 4 LX
Seems this latest patches included update from mennalox-mlnxen 5.4 to 5.9

I have no networking, and after an emergency network reset, eth0 and eth1 disappeared.

update1: used yum downgrade mellanox-mlnxen-5.4_1.0.3.0-4.xcpng8.3.x86_64
this didn't fix until I did another emergency network reset.

Looks like I'm back up and running.
Anyone else able to confirm this bug or am I alone in this hiccup?

I've done initial testing on Suse SLES/OpenSuse 15.4, 15.5 and the similar opensuse MicroOS 5.5

Installs, runs, reports correct information and stats so far. Basic commands work as expected.

@stormi No issues so far on Ryzen 1700x, Xeon e5-2640v3, and 2 Epyc 7313P systems

Reminder, Centos 7 EOL 1 year away

https://www.phoronix.com/news/CentOS-EOL-2023-7-Stream-8

@stormi
Here's a Ryzen 1700 on Asus TUF B550

Combined test results:
test-hvm32-selftest                      SUCCESS
test-hvm32pae-selftest                   SUCCESS
test-hvm32pse-selftest                   SUCCESS
test-hvm64-selftest                      SUCCESS
test-pv64-selftest                       SUCCESS
Combined test results:
test-pv64-cpuid-faulting                 SKIP
test-hvm32-umip                          SKIP
test-hvm64-umip                          SKIP
test-pv64-xsa-167                        SKIP
test-pv64-xsa-182                        SKIP
3

CPU Policy unit tests
Testing CPU vendor identification:
Testing CPUID serialise success:
Testing CPUID deserialise failure:
Testing CPUID out-of-range clearing:
Testing MSR serialise success:
Testing MSR deserialise failure:
Testing policy compatibility success:
Testing policy compatibility failure:
Done: all ok
0

@stormi Here's results from one of my lab servers. I'll run on some newer hardware next week.

Asus KGPE-D16, Opteron 6328, 128GB ram.

Combined test results:
test-hvm32-selftest                      SUCCESS
test-hvm32pae-selftest                   SUCCESS
test-hvm32pse-selftest                   SUCCESS
test-hvm64-selftest                      SUCCESS
test-pv64-selftest                       SUCCESS
Combined test results:
test-pv64-cpuid-faulting                 SKIP
test-pv64-pv-fsgsbase                    SKIP
test-hvm32-umip                          SKIP
test-hvm64-umip                          SKIP
test-pv64-xsa-167                        SKIP
test-pv64-xsa-182                        SKIP
3

CPU Policy unit tests
Testing CPU vendor identification:
Testing CPUID serialise success:
Testing CPUID deserialise failure:
Testing CPUID out-of-range clearing:
Testing MSR serialise success:
Testing MSR deserialise failure:
Testing policy compatibility success:
Testing policy compatibility failure:
Done: all ok
0

@cbaguzman Watchdog client service talks to ipmi module on hardware, or service on vm host if running virtualized, providing a heartbeat. If the heartbeat isn't received by the host, then a power cycle is initiated. In xen this would likely be via a xe vm-reset-powerstate command, but I haven't looked at the documentation

In short, watchdog works by host or hardware listening for the heartbeat, then hits a hard reset if doesn't hear it. All the guest does is send heartbeats to host watchdog server,
You can't expect your host to reboot itself if it's locked up, that's the point of it.