@TeddyAstie From what I've tried...It won't work at all in Linux. It kind of works in Windows, and while I do see some load on the GPU when running something, there's as much if not more load on the CPU. It's not what I expected at least. I've never really gamed on PC, and definitely never Linux. My only uses for GPU's have been in media composition which was always on dedicated systems and Nvidia GPUs. More recently I've been doing some AI tasks, again with passed-through NVidia GPUs. Side tangent... I ran into another workload last night where the app wanted OpenGL...But OpenGL doesn't work with the "normal" VGA adapter in XCP-ng, and I don't have a GPU on that particular pool to try a pass-through. That lead me to VirGL, but it doesn't look like that's gone anywhere.

@pierrebrunet said: Hi, just to be sure, are you saying it is working ok apart one task 8 days ago? Yes, after I adjusted the time it's working

@ravenet Sure, gave that a whirl and no change, though I did notice some nvtop weirdness as it would show some load, but most of the wait time there was actually no load on the GPUs instead of the constant load matching the model being loaded. For some historical context ARI support was initially disabled in the bios when I started this thread. That was on the list of things I enabled when I started seeing some success with ollama (something in the changes since has broken ollama now too, but there was at least some forward progress after enabling). dmesg output overall looked the same, but I did see this output on the console (and in dmesg) that seemed interesting. Not 100% sure at this point if this was in the previous dmesg outputs or not, but may be worth sharing. EDIT: looks like this may actually be new... I looked back through the past dm dmesg outputs and I did not see this output. [ 108.547683] amdgpu 0000:00:09.0: MES(0) failed to respond to msg=REMOVE_QUEUE [ 108.547729] amdgpu 0000:00:09.0: failed to remove hardware queue from MES, doorbell=0x1202 [ 108.547746] amdgpu 0000:00:09.0: MES might be in unrecoverable state, issue a GPU reset [ 108.547774] amdgpu 0000:00:09.0: Failed to evict queue 2 [ 108.547789] amdgpu 0000:00:09.0: Failed to evict process queues [ 108.547803] amdgpu: Failed to quiesce KFD [ 108.547870] amdgpu 0000:00:09.0: GPU reset begin!. Source: 3 [ 109.656850] amdgpu 0000:00:09.0: Failed to remove queue 0 [ 109.657324] amdgpu 0000:00:09.0: Dumping IP State [ 109.756265] amdgpu 0000:00:09.0: Dumping IP State Completed [ 112.047695] amdgpu 0000:00:09.0: MODE1 reset [ 112.047797] amdgpu 0000:00:09.0: GPU mode1 reset [ 112.054505] amdgpu 0000:00:09.0: GPU smu mode1 reset [ 113.075393] amdgpu 0000:00:09.0: GPU reset succeeded, trying to resume [ 113.090354] amdgpu 0000:00:09.0: [drm] PCIE GART of 512M enabled (table at 0x00000087D6B00000). [ 113.092905] amdgpu 0000:00:09.0: [drm] AMDGPU device coredump file has been created [ 113.092913] amdgpu 0000:00:09.0: [drm] Check your /sys/class/drm/card1/device/devcoredump/data [ 113.092917] amdgpu 0000:00:09.0: VRAM is lost due to GPU reset! [ 113.092921] amdgpu 0000:00:09.0: PSP is resuming... [ 114.997591] amdgpu 0000:00:09.0: GECC is disabled, set amdgpu_ras_enable=1 to enable GECC in next boot cycle if needed [ 115.091125] amdgpu 0000:00:09.0: RAP: optional rap ta ucode is not available [ 115.091130] amdgpu 0000:00:09.0: SECUREDISPLAY: optional securedisplay ta ucode is not available [ 115.091134] amdgpu 0000:00:09.0: SMU is resuming... [ 115.091375] amdgpu 0000:00:09.0: smu driver if version = 0x0000002e, smu fw if version = 0x00000033, smu fw program = 0, smu fw version = 0x00684c00 (104.76.0) [ 115.432371] amdgpu 0000:00:09.0: SMU is resumed successfully! [ 115.445556] amdgpu 0000:00:09.0: program CP_MES_CNTL : 0x4000000 [ 115.445731] amdgpu 0000:00:09.0: program CP_MES_CNTL : 0xc000000 [ 115.750240] amdgpu 0000:00:09.0: [drm] DMUB hardware initialized: version=0x0A000800 As an update for other things I have tried, in order to eliminate hardware issues, or bios settings I tried installing proxmox and spinning up a VM there with both GPUs passed through and it worked just fine... With that feedback I did a fresh install of XCP-NG 8.3 and spun up a fresh VM using the same steps as I used on proxmox and still no dice. This leads me to believe the issue is somewhere in the XCP-NG passthrough stack with my specific hardware...

@pedro_udifar Hi, You can try again with minPartSize=100000000 indeed. But I would advise you to contact B2 directly. They should have a clearer error to show instead of a 500. Thanks for your patience and your help

@christopher-petzel Thanks!

@MajorP93 No we do not have pro support at the moment hmmm. Really seems like a build with 9.2.19 could fix all of this. How can I request for a build @poddingue ? https://github.com/LINBIT/drbd/blob/drbd-9.2.19/ChangeLog#L61 9.2.17 Fix a kernel crash triggered by a crafted/invalid netlink message https://github.com/LINBIT/drbd/blob/drbd-9.2.19/ChangeLog#L24 9.2.19 Fix an AB-BA deadlock between online resize and activity-log transactions Fix several races during connection teardown that could crash or hang (ack_sender requeue, pending ping work, in-progress lb-tcp connect)

So if i set VMs to Best-effort or disabled for HA i get not enough memory. @pkgw Do you have Twinstore configured and/or HA enabled? If no I will continue my issues at the twinstore post. If i set to restart i get this.... host.setMaintenanceMode { "id": "35233210-4e37-4703-9bf6-9e8a9c24df9f", "maintenance": true } { "code": "HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN", "params": [], "call": { "duration": 5, "method": "host.disable", "params": [ "* session id *", "OpaqueRef:d9bd976f-445c-8244-1b56-879f446efc12" ] }, "message": "HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN()", "name": "XapiError", "stack": "XapiError: HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN() at XapiError.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12) at file:///opt/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21 at runNextTicks (node:internal/process/task_queues:65:5) at processImmediate (node:internal/timers:472:9)" }

@MajorP93 said: What you are describing must be a storage live migration then (XenMotion). That should work across pools, yes. I stand corrected because I did not factor in this variant in my initial answer. Yes, that's it - sorry - I should have been more explicit, even if 'lumbers along' wasn't a hint //EDIT: also there is no such thing as non-pool hosts. Even a single host setup has it's own pool. Pedantic, but true I technically meant 'not hosts in the source pool' I guess.

Got this issue since 2026-07-12.

@simonp , thank you very much and all of your team ! ! ! Keep up the outstanding work. Regards, Marc

@john.c Unfortunately, I cannot answer your question in the affirmative. I only use XCP-ng with TrueNAS. @Team-Storage have y'all had a chance to take a look at the bug report I filed? This issue is quite serious.

Your write-up is the kind of thing that saves the next person a weekend. One thing worth knowing before you commit to the whole-controller route. The passthrough page has a "Passing through Keyboards and Mice" section further down, and it says XCP-ng ships /etc/xensource/usb-policy.conf with DENY rules for mice and keyboards by default. You edit those to ALLOW, then refresh with /opt/xensource/libexec/usb_scan.py -d followed by xe pusb-scan host-uuid=<host_uuid>. It's at https://docs.xcp-ng.org/compute/ under USB Passthrough. I have no idea whether that covers your USB-to-serial adapter, which is a different device class, and passing the whole controller may still be the cleaner setup for two discrete workstations anyway. Might be worth a mention to @Team-Hypervisor-Kernel on the display question, because that one still puzzles me.

Indeed, it should be detected/reported correctly to avoid losing time finding the problem. Let me ping @julienXOVates

@Danp said: Smart Reboot option found on the host's Advanced tab does what you are asking Very nice!

@poddingue yeah I saw it now but I was more like mentioning that many people also have the same problem. Hope it gets resolved soon

@TrapoSAMA said: @john.c I have this iso but how to know what is de package? Thx It’s going to be an msi installer package or on the guest tools iso, which is on the XCP-ng or Xen Server your running likely in Local Storage SR or Shared Storage SR (if moved). Named the same as the file linked to above, likely. Anyway did you know that the UEFI SecureBoot certificates (at least one of them from 2011) issued by Microsoft are expiring fully during October 2026. Around that time likely at sometime after then those certificates will be placed into dbx. This means the Windows Server 2012 R2 if operating as a UEFI SecureBoot VM, will if restarted (or reboot following a crash), fail to boot. So your client will go from having a working app that they can use, but not upgrade at the moment to one which won’t work and can’t be used! They really need to upgrade or migrate from Windows to Linux (released during 2025 or 2026) along with the app if needed. Alternatively upgrade to Windows Server 2022 or 2025 and the app!

@the_jest Not showen in this picutre but this is where the message would be displayed. Next to the name of the host... [image: 1782931263879-screenshot-2026-07-01-144023.png]

For your storage question, it's fully explained in the doc: https://docs.xcp-ng.org/storage/#-how-to-modify-an-existing-sr-connection And yes, it's planned to get the complete error visible in XO, sadly, it's not "obvious" since the error message isn't returned by XAPI when you try to start but by another method we need to call after it fails ("assert can be started here" from the top of my head). Let me ping @julienXOvates

@poddingue Oh yes (dumb me). Very good, we'll see if they respond (as now there's two @ refferences). Happy Day