@Andrew thanks for all your hard work. I guess time will tell if the Vates team has to add those defaults to their official package or maybe I just got "lucky"

@Shrubbery said in xcp-ng 8.3 not recognizing all host memory: The issue was memory reserved for the integrated graphics card haha.. that is such a silly thing... but good to know then next time I'm working on some hardware and wondering where the RAM is going too...

@CodeMercenary Did some tests with the A2000 and as expected, the 12GB VRAM is the biggest limitation. Used vanilla installations of Ollama and ComfyUI with no tweaking or optimization. Especially in stable diffusion, the A2000 is about three times faster compared to the P40, but that is to be expected. I have added some results below. Stable Diffusion tests A2000 1024x1024, batch 1, iterations 30, cfg 4.0, euler 1.4s/it 1024x1024, batch 4, iterations 30, cfg 4.0, euler 2.5s/it = 0.6s/it P40 1024x1024, batch 1, iterations 30, cfg 4.0, euler 2.8s/it 1024x1024, batch 4, iterations 30, cfg 4.0, euler 12.1s/it = 3s/it Inference tests A2000 qwen2.5:14b 21 token/sec qwen2.5-coder:14b 21 token/sec llama3.2:3b-Q8 50 token/sec llama3.2:3b-Q4 60 token/sec P40 qwen2.5:14b 17 token/sec qwen2.5-coder:14b 17 token/sec llama3.2:3b-Q8 40 token/sec llama3.2:3b-Q4 48 token/sec During heavy testing, the A2000 reached 70°C and the P40 reached 60°C both with the Dell R720 set to automatic fan control.

@steff22 said in Gpu passthrough on Asrock rack B650D4U3-2L2Q will not work: @ravenet Ok I think the pro gpu's work better without so many bugs. I switched to Nvidia RTX 4070. With the Nvidia gpu and this works as it should and gets an image on the screen on the first try. So there must be something wrong with the amd drivers plus that error with the amd reset bug I've setup a test system with a ryzen 7700x and a regular radeon 7600xt, non-pro gpu. Will let you know results. Shouldn't be an issue, it's same hardware and even the drivers work across them. Though they do have different bios on cards

@Andrew Thanks Andrew sounds good , let me know when it suits..

@nick-lloyd thanks, I did! Passthrough for AMD consumer Hardare iGPUs seams not to properly function… or, I should say, the Linux VMs are experiencing troubles correctly dealing with it: the devices are correctly passing down to the VM, but for some reason the ROCm kernel module is not capable of dealing with them. Could it be be ause it expected them to be on a specific bus? (They are assigned a different bus id) Could it be because it expected them to be under the same bus id? (They are scattered around different dedicated bus ids) Not sure, but the end result is the iGPU cannot be correctly initialized, albeit detected.

@gduperrey (unrelated to this fan issue) I loaded the new standard 8.2 testing kernel on my NUC11 and it seems to boot a little faster and also no longer complains about some APIC devices.

Do we have a list of GPUs that actually do support vGPU (not Passthrough) in 8.3? The documentation seems to be lagging behind there, a lot of references to the AMD mxgpu setups which have been deprecated.

Ah indeed it makes sense then

@imaginapix Didn't think of that.... But I guess it's more work than it's worth in this instance.

@manilx True, not a big deal, but this two host were fresh installs with 8.3 release and only joint the pool to keep everything running. So this seems to be a thing on pool level, since on those hosts only the Legrand was ever attached to.

I have been able to run a script at startup that enables 2.5g on the Intel X550. Here's how : Create the script that will run at startup nano /usr/local/bin/x550.sh Content of x550.sh : #!/usr/bin/bash sleep 1m ethtool -s eth1 advertise 0x1800000001028 echo `date +%F" "%T` "Startup script worked" >> /root/x550.log Make the script executable : chmod +x /usr/local/bin/x550.sh Create the service that runs the script at startup nano /etc/systemd/system/x550.service Content of x550.service : [Unit] Description=Runs /usr/local/bin/x550.sh [Service] ExecStart=/usr/local/bin/x550.sh [Install] WantedBy=multi-user.target Enable the service systemctl enable x550 Reboot and run this command to see if the link speed is 2.5g ethtool ethX (replace X with yours) It should show this result : Supported ports: [ TP ] Supported link modes: 100baseT/Full 1000baseT/Full 10000baseT/Full 2500baseT/Full 5000baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/Full 2500baseT/Full 5000baseT/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Speed: 2500Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on MDI-X: Unknown Supports Wake-on: d Wake-on: d Current message level: 0x00000007 (7) drv probe link Link detected: yes You can check the log file of your script located at /root/x550.log I used this article to create everything : https://www.redhat.com/sysadmin/replacing-rclocal-systemd

@olivierlambert Thank you - appreciate for the confirmation.

@olivierlambert said in Kernel trap (??) booting TrueNAS with 2 x Kingston NVMe SSDs: Not all NVMe are created equals Now all I need to do is determine if these errors are from the NVME's themselves or the 4 x NVMe sled that they're inserted in. LOL. Again, thanks for the help.

@TS79 I had not seen that adapter, finding a main board that supports bifurcation is probably easier than a card that bifurcates. I'll have to look at some of the devices I have and see if this is supported.

Ok so I'm going to answer my own question. The answer is that the vdi (local SR) you are trying to boot from and the pci slots are on two different machines and you need to migrate the boot vdi to a shared pool sr or to the local sr of the machine with the pci slots!

No worries, I had the same problem few months ago

Thank you, low on sleep and of course you were right...