@gskger said in NVIDIA Tesla M40 for AI work in Ubuntu VM - is it a good idea?:

Nvidia RTX A2000 12GB

I am curious how your testing goes. That sounds like a great card. Not as expensive as the T4 so might be more reasonable for me to consider.

@steff22 said in Gpu passthrough on Asrock rack B650D4U3-2L2Q will not work:

@ravenet Ok I think the pro gpu's work better without so many bugs.

I switched to Nvidia RTX 4070. With the Nvidia gpu and this works as it should and gets an image on the screen on the first try.

So there must be something wrong with the amd drivers plus that error with the amd reset bug

I've setup a test system with a ryzen 7700x and a regular radeon 7600xt, non-pro gpu. Will let you know results. Shouldn't be an issue, it's same hardware and even the drivers work across them. Though they do have different bios on cards

@Andrew Thanks Andrew sounds good , let me know when it suits..

@nick-lloyd thanks, I did! Passthrough for AMD consumer Hardare iGPUs seams not to properly function… or, I should say, the Linux VMs are experiencing troubles correctly dealing with it: the devices are correctly passing down to the VM, but for some reason the ROCm kernel module is not capable of dealing with them. Could it be be ause it expected them to be on a specific bus? (They are assigned a different bus id)
Could it be because it expected them to be under the same bus id? (They are scattered around different dedicated bus ids)
Not sure, but the end result is the iGPU cannot be correctly initialized, albeit detected.

@MasterOSkillio said in Intel CPUs with P and E cores or other server CPU suggestions:

From what I have seen in various places on the web P and E cores are not really supported yet for XCP-NG? Is it safe to assume that is changing? I believe proxmox, and VMware have moved to support them. If I use the 12600K will I have to turn off one (P or E) and use only the other?

Not sure what you mean by "support" but I'm running as of now the P+E core setup and it works just fine. Stable, performs as expected.

@gduperrey (unrelated to this fan issue) I loaded the new standard 8.2 testing kernel on my NUC11 and it seems to boot a little faster and also no longer complains about some APIC devices.

Regarding memory test: Just running the normal mem test from Grub should do, I guess?

Do we have a list of GPUs that actually do support vGPU (not Passthrough) in 8.3?

The documentation seems to be lagging behind there, a lot of references to the AMD mxgpu setups which have been deprecated.

@olivierlambert Correct.

When switching to the virtual console through iDRAC it works.

Is there a way to set the resolution of the OS when it starts and goes into xsconsole?

Same thing happens on the OS installer as well. Maybe the resolution is too low for the screen.

Ah indeed it makes sense then 🙂

Hi Teddy,
thanks for the analysis and your brief explanation.
IOMMU should be properly activated:

(XEN) [ 0.221843] AMD-Vi: IOMMU Extended Features:
(XEN) [ 0.222606] - Peripheral Page Service Request
(XEN) [ 0.223366] - NX bit
(XEN) [ 0.224123] - Guest APIC Physical Processor Interrupt
(XEN) [ 0.224889] - Invalidate All Command
(XEN) [ 0.225649] - Guest APIC
(XEN) [ 0.226412] - Performance Counters
(XEN) [ 0.227178] - Host Address Translation Size: 0x2
(XEN) [ 0.227940] - Guest Address Translation Size: 0
(XEN) [ 0.228681] - Guest CR3 Root Table Level: 0x1
(XEN) [ 0.229416] - Maximum PASID: 0xf
(XEN) [ 0.230140] - SMI Filter Register: 0x1
(XEN) [ 0.230867] - SMI Filter Register Count: 0x1
(XEN) [ 0.231596] - Guest Virtual APIC Modes: 0x1
(XEN) [ 0.232316] - Dual PPR Log: 0x2
(XEN) [ 0.233024] - Dual Event Log: 0x2
(XEN) [ 0.233727] - Secure ATS
(XEN) [ 0.234424] - User / Supervisor Page Protection
(XEN) [ 0.235126] - Device Table Segmentation: 0x3
(XEN) [ 0.235826] - PPR Log Overflow Early Warning
(XEN) [ 0.236514] - PPR Automatic Response
(XEN) [ 0.237198] - Memory Access Routing and Control: 0x1
(XEN) [ 0.237881] - Block StopMark Message
(XEN) [ 0.238558] - Performance Optimization
(XEN) [ 0.239234] - MSI Capability MMIO Access
(XEN) [ 0.239906] - Guest I/O Protection
(XEN) [ 0.240570] - Enhanced PPR Handling
(XEN) [ 0.241231] - Invalidate IOTLB Type
(XEN) [ 0.241886] - VM Table Size: 0x2
(XEN) [ 0.242537] - Guest Access Bit Update Disable
(XEN) [ 0.252988] AMD-Vi: IOMMU 0 Enabled.
(XEN) [ 0.253820] I/O virtualisation enabled

I have also read about the problem with UEFI several times.
Now I am going to try to find a system with "legacy" boot mode.

@knut99 Note for the UH125, it uses the RTL8126 ethernet chip. While it may work with XCP, it's officially not supported due to the Realtek drivers.

@imaginapix Didn't think of that.... But I guess it's more work than it's worth in this instance.

@manilx

True, not a big deal, but this two host were fresh installs with 8.3 release and only joint the pool to keep everything running.

So this seems to be a thing on pool level, since on those hosts only the Legrand was ever attached to.

I have been able to run a script at startup that enables 2.5g on the Intel X550. Here's how :

Create the script that will run at startup

nano /usr/local/bin/x550.sh

Content of x550.sh :

#!/usr/bin/bash sleep 1m ethtool -s eth1 advertise 0x1800000001028 echo `date +%F" "%T` "Startup script worked" >> /root/x550.log

Make the script executable :

chmod +x /usr/local/bin/x550.sh

Create the service that runs the script at startup

nano /etc/systemd/system/x550.service

Content of x550.service :

[Unit] Description=Runs /usr/local/bin/x550.sh [Service] ExecStart=/usr/local/bin/x550.sh [Install] WantedBy=multi-user.target

Enable the service

systemctl enable x550

Reboot and run this command to see if the link speed is 2.5g

ethtool ethX (replace X with yours)

It should show this result :

Supported ports: [ TP ] Supported link modes: 100baseT/Full 1000baseT/Full 10000baseT/Full 2500baseT/Full 5000baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/Full 2500baseT/Full 5000baseT/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Speed: 2500Mb/s Duplex: Full Port: Twisted Pair PHYAD: 0 Transceiver: internal Auto-negotiation: on MDI-X: Unknown Supports Wake-on: d Wake-on: d Current message level: 0x00000007 (7) drv probe link Link detected: yes

You can check the log file of your script located at /root/x550.log

I used this article to create everything :
https://www.redhat.com/sysadmin/replacing-rclocal-systemd

@olivierlambert Thank you - appreciate for the confirmation.

@olivierlambert said in Kernel trap (??) booting TrueNAS with 2 x Kingston NVMe SSDs:

Not all NVMe are created equals

Now all I need to do is determine if these errors are from the NVME's themselves or the 4 x NVMe sled that they're inserted in. LOL.

Again, thanks for the help.

@TS79

I had not seen that adapter, finding a main board that supports bifurcation is probably easier than a card that bifurcates. I'll have to look at some of the devices I have and see if this is supported.

@Greg_E Fans and hard drives eat up more power than most people expect.