[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.8.0-134-generic root=UUID=cc6f3ac6-f24d-4276-abe9-c242adec4e04 ro amdgpu.msi=0 console=tty1 console=ttyS0 [ 0.073343] Kernel command line: BOOT_IMAGE=/vmlinuz-6.8.0-134-generic root=UUID=cc6f3ac6-f24d-4276-abe9-c242adec4e04 ro amdgpu.msi=0 console=tty1 console=ttyS0 [ 1.945574] smp: Bringing up secondary CPUs ... [ 2.326535] ACPI: PM: Registering ACPI NVS region [mem 0xfc000000-0xfc00afff] (45056 bytes) [ 2.546441] PCI: Ignoring E820 reservations for host bridge windows [ 5.945544] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 5.979355] Initialise system trusted keyrings [ 6.027466] integrity: Platform Keyring initialized [ 6.035801] integrity: Machine keyring initialized [ 6.195357] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled [ 6.497076] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 7.090375] usb 1-2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 11.694811] [drm] amdgpu kernel modesetting enabled. [ 11.698198] [drm] amdgpu version: 6.16.13 [ 11.707244] amdgpu: Virtual CRAT table created for CPU [ 11.712863] amdgpu: Topology: Add CPU node [ 11.723315] amdgpu 0000:00:08.0: amdgpu: initializing kernel modesetting (IP DISCOVERY 0x1002:0x7551 0x1849:0x5413 0xC0). [ 11.733858] amdgpu 0000:00:08.0: amdgpu: register mmio base: 0xE0800000 [ 11.739199] amdgpu 0000:00:08.0: amdgpu: register mmio size: 524288 [ 11.751074] amdgpu 0000:00:08.0: amdgpu: detected ip block number 0 (soc24_common) [ 11.758865] amdgpu 0000:00:08.0: amdgpu: detected ip block number 1 (gmc_v12_0) [ 11.766084] amdgpu 0000:00:08.0: amdgpu: detected ip block number 2 (ih_v7_0) [ 11.773082] amdgpu 0000:00:08.0: amdgpu: detected ip block number 3 (psp) [ 11.779859] amdgpu 0000:00:08.0: amdgpu: detected ip block number 4 (smu) [ 11.786657] amdgpu 0000:00:08.0: amdgpu: detected ip block number 5 (dm) [ 11.793372] amdgpu 0000:00:08.0: amdgpu: detected ip block number 6 (gfx_v12_0) [ 11.800716] amdgpu 0000:00:08.0: amdgpu: detected ip block number 7 (sdma_v7_0) [ 11.807886] amdgpu 0000:00:08.0: amdgpu: detected ip block number 8 (vcn_v5_0_0) [ 11.815207] amdgpu 0000:00:08.0: amdgpu: detected ip block number 9 (jpeg_v5_0_0) [ 11.823661] amdgpu 0000:00:08.0: amdgpu: detected ip block number 10 (mes_v12_0) [ 11.837090] amdgpu 0000:00:08.0: Invalid PCI ROM data signature: expecting 0x52494350, got 0xe94faa55 [ 11.877514] amdgpu 0000:00:08.0: amdgpu: Fetched VBIOS from ROM [ 11.883418] amdgpu: ATOM BIOS: 113-APM107573-101 [ 11.896900] amdgpu 0000:00:08.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported [ 11.904766] amdgpu 0000:00:08.0: amdgpu: MEM ECC is active. [ 11.910263] amdgpu 0000:00:08.0: amdgpu: SRAM ECC is not presented. [ 11.916412] amdgpu 0000:00:08.0: amdgpu: RAS INFO: ras initialized successfully, hardware ability[101] ras_mask[101] [ 11.925907] amdgpu 0000:00:08.0: amdgpu: vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit [ 11.935410] amdgpu 0000:00:08.0: amdgpu: System can't access extended configuration space, please check!! [ 11.944064] amdgpu 0000:00:08.0: amdgpu: VRAM: 32624M 0x0000008000000000 - 0x00000087F6FFFFFF (32624M used) [ 11.952987] amdgpu 0000:00:08.0: amdgpu: GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF [ 11.970767] amdgpu 0000:00:08.0: amdgpu: amdgpu: 32624M of VRAM memory ready [ 11.977495] amdgpu 0000:00:08.0: amdgpu: amdgpu: 16044M of GTT memory ready. [ 11.990332] amdgpu 0000:00:08.0: amdgpu: PCIE GART of 512M enabled (table at 0x00000087D6B00000). [ 12.000393] amdgpu 0000:00:08.0: amdgpu: [drm] Loading DMUB firmware via PSP: version=0x0A000601 [ 12.008984] amdgpu 0000:00:08.0: amdgpu: [VCN instance 0] Found VCN firmware Version ENC: 1.11 DEC: 9 VEP: 0 Revision: 1 [ 12.258087] amdgpu 0000:00:08.0: amdgpu: GECC is disabled, set amdgpu_ras_enable=1 to enable GECC in next boot cycle if needed [ 12.278761] amdgpu 0000:00:08.0: amdgpu: RAP: optional rap ta ucode is not available [ 12.286538] amdgpu 0000:00:08.0: amdgpu: SECUREDISPLAY: optional securedisplay ta ucode is not available [ 12.295298] amdgpu 0000:00:08.0: amdgpu: smu driver if version = 0x0000002e, smu fw if version = 0x00000032, smu fw program = 0, smu fw version = 0x00684b00 (104.75.0) [ 12.308986] amdgpu 0000:00:08.0: amdgpu: SMU driver if version not matched [ 12.446679] amdgpu 0000:00:08.0: amdgpu: SMU is initialized successfully! [ 12.461005] amdgpu 0000:00:08.0: amdgpu: [drm] Display Core v3.2.359 initialized on DCN 4.0.1 [ 12.467798] amdgpu 0000:00:08.0: amdgpu: [drm] DP-HDMI FRL PCON supported [ 12.549593] amdgpu 0000:00:08.0: amdgpu: [drm] DMUB hardware initialized: version=0x0A000601 [ 12.646207] amdgpu 0000:00:08.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 12.659059] amdgpu 0000:00:08.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 12.671193] amdgpu 0000:00:08.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 12.683996] amdgpu 0000:00:08.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 12.714590] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0x4000000 [ 12.720527] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0xc000000 [ 12.727344] amdgpu 0000:00:08.0: amdgpu: MES FW version must be >= 0x82 to enable LR compute workaround. [ 13.032286] amdgpu: HMM registered 32624MB device memory [ 13.045208] kfd kfd: amdgpu: Allocated 3969056 bytes on gart [ 13.050832] kfd kfd: amdgpu: Total number of KFD nodes to be created: 1 [ 13.056802] amdgpu: Virtual CRAT table created for GPU [ 13.062063] amdgpu: Topology: Add dGPU node [0x7551:0x1002] [ 13.379214] amdgpu 0000:00:08.0: added peer-to-peer DMA memory 0x1000000000-0x17ffffffff [ 13.384871] amdgpu 0000:00:08.0: amdgpu: AIS: registered 32624MB device memory [ 13.389980] kfd kfd: amdgpu: added device 1002:7551 [ 13.394048] amdgpu 0000:00:08.0: amdgpu: SE 4, SH per SE 2, CU per SH 8, active_cu_number 64 [ 13.399924] amdgpu 0000:00:08.0: amdgpu: ring gfx_0.0.0 uses VM inv eng 0 on hub 0 [ 13.407688] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0 [ 13.414059] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0 [ 13.419947] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.1 uses VM inv eng 6 on hub 0 [ 13.425700] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.1 uses VM inv eng 7 on hub 0 [ 13.431553] amdgpu 0000:00:08.0: amdgpu: ring sdma0 uses VM inv eng 8 on hub 0 [ 13.437301] amdgpu 0000:00:08.0: amdgpu: ring sdma1 uses VM inv eng 9 on hub 0 [ 13.444101] amdgpu 0000:00:08.0: amdgpu: ring vcn_unified_0 uses VM inv eng 0 on hub 8 [ 13.451797] amdgpu 0000:00:08.0: amdgpu: ring jpeg_dec uses VM inv eng 1 on hub 8 [ 13.523212] amdgpu 0000:00:08.0: amdgpu: Runtime PM not available [ 13.531517] [drm] Initialized amdgpu 3.64.0 20150101 for 0000:00:08.0 on minor 1 [ 13.585710] amdgpu 0000:00:08.0: [drm] Cannot find any crtc or sizes [ 13.601031] amdgpu 0000:00:09.0: amdgpu: initializing kernel modesetting (IP DISCOVERY 0x1002:0x7551 0x1849:0x5413 0xC0). [ 13.612291] amdgpu 0000:00:08.0: [drm] Cannot find any crtc or sizes [ 13.615140] amdgpu 0000:00:09.0: amdgpu: register mmio base: 0xE0880000 [ 13.618072] amdgpu 0000:00:08.0: [drm] Cannot find any crtc or sizes [ 13.626605] amdgpu 0000:00:09.0: amdgpu: register mmio size: 524288 [ 13.632989] amdgpu 0000:00:09.0: amdgpu: detected ip block number 0 (soc24_common) [ 13.654934] amdgpu 0000:00:09.0: amdgpu: detected ip block number 1 (gmc_v12_0) [ 13.662701] amdgpu 0000:00:09.0: amdgpu: detected ip block number 2 (ih_v7_0) [ 13.669974] amdgpu 0000:00:09.0: amdgpu: detected ip block number 3 (psp) [ 13.677202] amdgpu 0000:00:09.0: amdgpu: detected ip block number 4 (smu) [ 13.684417] amdgpu 0000:00:09.0: amdgpu: detected ip block number 5 (dm) [ 13.691457] amdgpu 0000:00:09.0: amdgpu: detected ip block number 6 (gfx_v12_0) [ 13.699055] amdgpu 0000:00:09.0: amdgpu: detected ip block number 7 (sdma_v7_0) [ 13.706531] amdgpu 0000:00:09.0: amdgpu: detected ip block number 8 (vcn_v5_0_0) [ 13.714049] amdgpu 0000:00:09.0: amdgpu: detected ip block number 9 (jpeg_v5_0_0) [ 13.721589] amdgpu 0000:00:09.0: amdgpu: detected ip block number 10 (mes_v12_0) [ 13.729241] amdgpu 0000:00:09.0: Invalid PCI ROM data signature: expecting 0x52494350, got 0xe94faa55 [ 13.754294] amdgpu 0000:00:09.0: amdgpu: Fetched VBIOS from ROM [ 13.760013] amdgpu: ATOM BIOS: 113-APM107573-101 [ 13.771765] amdgpu 0000:00:09.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported [ 13.779437] amdgpu 0000:00:09.0: amdgpu: MEM ECC is active. [ 13.784650] amdgpu 0000:00:09.0: amdgpu: SRAM ECC is not presented. [ 13.790455] amdgpu 0000:00:09.0: amdgpu: RAS INFO: ras initialized successfully, hardware ability[101] ras_mask[101] [ 13.799699] amdgpu 0000:00:09.0: amdgpu: vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit [ 13.808860] amdgpu 0000:00:09.0: amdgpu: System can't access extended configuration space, please check!! [ 13.817357] amdgpu 0000:00:09.0: amdgpu: VRAM: 32624M 0x0000008000000000 - 0x00000087F6FFFFFF (32624M used) [ 13.826029] amdgpu 0000:00:09.0: amdgpu: GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF [ 13.842967] amdgpu 0000:00:09.0: amdgpu: amdgpu: 32624M of VRAM memory ready [ 13.849564] amdgpu 0000:00:09.0: amdgpu: amdgpu: 16044M of GTT memory ready. [ 13.862071] amdgpu 0000:00:09.0: amdgpu: PCIE GART of 512M enabled (table at 0x00000087D6B00000). [ 13.871847] amdgpu 0000:00:09.0: amdgpu: [drm] Loading DMUB firmware via PSP: version=0x0A000601 [ 13.880216] amdgpu 0000:00:09.0: amdgpu: [VCN instance 0] Found VCN firmware Version ENC: 1.11 DEC: 9 VEP: 0 Revision: 1 [ 14.106808] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 14.114753] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 14.130133] amdgpu 0000:00:09.0: amdgpu: GECC is disabled, set amdgpu_ras_enable=1 to enable GECC in next boot cycle if needed [ 14.146175] amdgpu 0000:00:09.0: amdgpu: RAP: optional rap ta ucode is not available [ 14.153238] amdgpu 0000:00:09.0: amdgpu: SECUREDISPLAY: optional securedisplay ta ucode is not available [ 14.161458] amdgpu 0000:00:09.0: amdgpu: smu driver if version = 0x0000002e, smu fw if version = 0x00000032, smu fw program = 0, smu fw version = 0x00684b00 (104.75.0) [ 14.174606] amdgpu 0000:00:09.0: amdgpu: SMU driver if version not matched [ 14.309355] amdgpu 0000:00:09.0: amdgpu: SMU is initialized successfully! [ 14.323620] amdgpu 0000:00:09.0: amdgpu: [drm] Display Core v3.2.359 initialized on DCN 4.0.1 [ 14.330138] amdgpu 0000:00:09.0: amdgpu: [drm] DP-HDMI FRL PCON supported [ 14.411949] amdgpu 0000:00:09.0: amdgpu: [drm] DMUB hardware initialized: version=0x0A000601 [ 14.502383] amdgpu 0000:00:09.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 14.514610] amdgpu 0000:00:09.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 14.526608] amdgpu 0000:00:09.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 14.538686] amdgpu 0000:00:09.0: amdgpu: [drm] PSR support 0, DC PSR ver -1, sink PSR ver 0 DPCD caps 0x0 su_y_granularity 0 [ 14.567058] amdgpu 0000:00:09.0: amdgpu: program CP_MES_CNTL : 0x4000000 [ 14.572376] amdgpu 0000:00:09.0: amdgpu: program CP_MES_CNTL : 0xc000000 [ 14.610747] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 14.618746] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 14.795314] amdgpu: HMM registered 32624MB device memory [ 14.804558] kfd kfd: amdgpu: Allocated 3969056 bytes on gart [ 14.809095] kfd kfd: amdgpu: Total number of KFD nodes to be created: 1 [ 14.814246] amdgpu: Virtual CRAT table created for GPU [ 14.819371] amdgpu: Topology: Add dGPU node [0x7551:0x1002] [ 15.168251] amdgpu 0000:00:09.0: added peer-to-peer DMA memory 0x1800000000-0x1fffffffff [ 15.174595] amdgpu 0000:00:09.0: amdgpu: AIS: registered 32624MB device memory [ 15.180173] kfd kfd: amdgpu: added device 1002:7551 [ 15.184597] amdgpu 0000:00:09.0: amdgpu: SE 4, SH per SE 2, CU per SH 8, active_cu_number 64 [ 15.191951] amdgpu 0000:00:09.0: amdgpu: ring gfx_0.0.0 uses VM inv eng 0 on hub 0 [ 15.200136] amdgpu 0000:00:09.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0 [ 15.207926] amdgpu 0000:00:09.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0 [ 15.215634] amdgpu 0000:00:09.0: amdgpu: ring comp_1.0.1 uses VM inv eng 6 on hub 0 [ 15.223289] amdgpu 0000:00:09.0: amdgpu: ring comp_1.1.1 uses VM inv eng 7 on hub 0 [ 15.230935] amdgpu 0000:00:09.0: amdgpu: ring sdma0 uses VM inv eng 8 on hub 0 [ 15.238334] amdgpu 0000:00:09.0: amdgpu: ring sdma1 uses VM inv eng 9 on hub 0 [ 15.245646] amdgpu 0000:00:09.0: amdgpu: ring vcn_unified_0 uses VM inv eng 0 on hub 8 [ 15.253462] amdgpu 0000:00:09.0: amdgpu: ring jpeg_dec uses VM inv eng 1 on hub 8 [ 15.323746] amdgpu 0000:00:09.0: amdgpu: Runtime PM not available [ 15.331766] [drm] Initialized amdgpu 3.64.0 20150101 for 0000:00:09.0 on minor 2 [ 15.384945] amdgpu 0000:00:09.0: [drm] Cannot find any crtc or sizes [ 15.391880] amdgpu 0000:00:09.0: [drm] Cannot find any crtc or sizes [ 15.898799] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 15.906927] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 16.402788] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 16.410788] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 18.050986] systemd[1]: Starting lvm2-monitor.service - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling... [ 19.360810] snd_hda_intel 0000:00:06.0: bound 0000:00:08.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu]) [ 19.368886] snd_hda_intel 0000:00:07.0: bound 0000:00:09.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu]) [ 24.516563] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. [ 247.225604] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 247.729618] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 247.729663] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 248.233599] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 248.241613] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 248.745624] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 248.753606] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 249.257727] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 249.353593] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 249.857587] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 250.361583] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 250.865559] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 251.369581] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 251.873578] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 252.417555] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 252.921550] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 253.425544] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 253.929532] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 254.433545] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 254.937513] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 261.025433] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 261.033434] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 261.041433] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 261.529426] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 261.537425] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 261.545442] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 262.161425] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 262.666402] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 262.666417] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 263.097452] amdgpu 0000:00:08.0: amdgpu: Dumping IP State [ 263.170431] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 263.193516] amdgpu 0000:00:08.0: amdgpu: Dumping IP State Completed [ 263.193592] amdgpu 0000:00:08.0: amdgpu: [drm] AMDGPU device coredump file has been created [ 263.193596] amdgpu 0000:00:08.0: amdgpu: [drm] Check your /sys/class/drm/card1/device/devcoredump/data [ 263.193599] amdgpu 0000:00:08.0: amdgpu: ring sdma0 timeout, signaled seq=52, emitted seq=54 [ 263.201313] amdgpu 0000:00:08.0: amdgpu: Starting sdma0 ring reset [ 263.211857] amdgpu 0000:00:08.0: amdgpu: reset sdma queue (0:0:0) [ 263.327520] amdgpu 0000:00:08.0: [drm:amdgpu_ring_test_helper [amdgpu]] *ERROR* ring sdma0 test failed (-110) [ 263.335538] amdgpu 0000:00:08.0: amdgpu: Ring sdma0 reset failed [ 263.340082] amdgpu 0000:00:08.0: amdgpu: GPU reset begin!. Source: 1 [ 263.341423] amdgpu 0000:00:08.0: amdgpu: Failed to evict queue 2 [ 263.351160] amdgpu 0000:00:08.0: amdgpu: remove_all_kfd_queues_mes: Failed to remove queue 1 for dev 27203 [ 265.632156] amdgpu 0000:00:08.0: amdgpu: MODE1 reset [ 265.632259] amdgpu 0000:00:08.0: amdgpu: GPU mode1 reset [ 265.638923] amdgpu 0000:00:08.0: amdgpu: GPU smu mode1 reset [ 266.579395] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.580089] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.580784] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.581443] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.582129] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.582825] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.583544] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.584230] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.584928] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.585625] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.586311] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.587009] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.587704] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.588437] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.589124] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.589819] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.590548] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.591234] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.591933] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.592628] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.593314] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.594013] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.594710] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.595440] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.596129] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.596826] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.597547] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.598232] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.598930] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.599625] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.600311] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.601009] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.601220] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.601916] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.602612] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.603297] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.603996] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.604691] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.605437] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.606125] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.606822] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.607550] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.608238] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.608937] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.609632] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.610318] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.611017] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.611714] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.612438] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.613126] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.613824] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.614547] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.615233] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.615932] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.616627] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.617313] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.618012] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.618708] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.619437] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.620124] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.620821] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.621544] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.622229] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.622927] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.623131] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.623826] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.624550] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.625231] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.625927] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.626623] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.627309] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.628004] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.628700] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.629438] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.630126] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.630824] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.631546] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.632234] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.632932] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.633628] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.634315] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.635013] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.635709] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.636441] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.637129] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.637823] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.638548] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.639233] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.639932] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.640626] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.641312] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.642010] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.642705] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.643438] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.644125] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.644824] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.645028] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.645724] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.646442] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.647128] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.647821] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.648474] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.649165] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.650003] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.650727] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.651687] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.652687] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.653445] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.654135] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.654833] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.655546] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.656233] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.656931] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.657626] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.658312] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.659009] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.659705] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.660437] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.661124] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.661821] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.662544] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.663229] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.663928] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.664623] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.665309] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.666008] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.666705] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.667435] amdgpu 0000:00:08.0: amdgpu: [drm] REG_WAIT timeout 1us * 11 tries - acquire_engine line:146 [ 266.688969] amdgpu 0000:00:08.0: amdgpu: GPU reset succeeded, trying to resume [ 266.704040] amdgpu 0000:00:08.0: amdgpu: PCIE GART of 512M enabled (table at 0x00000087D6B00000). [ 266.706652] amdgpu 0000:00:08.0: amdgpu: VRAM is lost due to GPU reset! [ 266.706660] amdgpu 0000:00:08.0: amdgpu: PSP is resuming... [ 268.586778] amdgpu 0000:00:08.0: amdgpu: GECC is disabled, set amdgpu_ras_enable=1 to enable GECC in next boot cycle if needed [ 268.680554] amdgpu 0000:00:08.0: amdgpu: RAP: optional rap ta ucode is not available [ 268.680558] amdgpu 0000:00:08.0: amdgpu: SECUREDISPLAY: optional securedisplay ta ucode is not available [ 268.680562] amdgpu 0000:00:08.0: amdgpu: SMU is resuming... [ 268.680767] amdgpu 0000:00:08.0: amdgpu: smu driver if version = 0x0000002e, smu fw if version = 0x00000032, smu fw program = 0, smu fw version = 0x00684b00 (104.75.0) [ 268.680771] amdgpu 0000:00:08.0: amdgpu: SMU driver if version not matched [ 269.027009] amdgpu 0000:00:08.0: amdgpu: SMU is resumed successfully! [ 269.040134] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0x4000000 [ 269.040346] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0xc000000 [ 269.344952] amdgpu 0000:00:08.0: amdgpu: [drm] DMUB hardware initialized: version=0x0A000601 [ 269.436452] amdgpu 0000:00:08.0: amdgpu: ring gfx_0.0.0 uses VM inv eng 0 on hub 0 [ 269.436457] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0 [ 269.436459] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0 [ 269.436461] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.1 uses VM inv eng 6 on hub 0 [ 269.436463] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.1 uses VM inv eng 7 on hub 0 [ 269.436465] amdgpu 0000:00:08.0: amdgpu: ring sdma0 uses VM inv eng 8 on hub 0 [ 269.436467] amdgpu 0000:00:08.0: amdgpu: ring sdma1 uses VM inv eng 9 on hub 0 [ 269.436468] amdgpu 0000:00:08.0: amdgpu: ring vcn_unified_0 uses VM inv eng 0 on hub 8 [ 269.436470] amdgpu 0000:00:08.0: amdgpu: ring jpeg_dec uses VM inv eng 1 on hub 8 [ 269.527630] amdgpu 0000:00:08.0: amdgpu: GPU reset(1) succeeded! [ 269.527650] amdgpu 0000:00:08.0: [drm] device wedged, but recovered through reset [ 270.057341] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 270.073301] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 270.562315] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 270.577299] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 270.577300] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 271.065335] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 271.081293] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 271.089294] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 271.569293] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 271.593305] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 271.601288] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 272.073311] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 272.105297] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 272.577312] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 273.081309] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 273.585295] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 274.089248] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 400.823796] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 401.359725] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 401.863724] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 402.367730] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 402.871735] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 403.375699] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 403.879691] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 404.383682] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 404.887740] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 405.391690] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 405.895675] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 406.399673] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 427.765333] workqueue: amdgpu_tlb_fence_work [amdgpu] hogged CPU for >10000us 4 times, consider switching to WQ_UNBOUND [ 428.033330] workqueue: amdgpu_tlb_fence_work [amdgpu] hogged CPU for >10000us 8 times, consider switching to WQ_UNBOUND [ 428.581324] workqueue: amdgpu_tlb_fence_work [amdgpu] hogged CPU for >10000us 16 times, consider switching to WQ_UNBOUND [ 492.286636] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 492.302612] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 492.791628] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 492.806629] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 492.806694] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 493.310640] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 493.814699] amdgpu 0000:00:08.0: amdgpu: Dumping IP State [ 493.911250] amdgpu 0000:00:08.0: amdgpu: Dumping IP State Completed [ 493.911267] amdgpu 0000:00:08.0: amdgpu: [drm] AMDGPU device coredump file has been created [ 493.911270] amdgpu 0000:00:08.0: amdgpu: [drm] Check your /sys/class/drm/card1/device/devcoredump/data [ 493.911272] amdgpu 0000:00:08.0: amdgpu: ring sdma0 timeout, signaled seq=135, emitted seq=137 [ 493.918393] amdgpu 0000:00:08.0: amdgpu: Starting sdma0 ring reset [ 493.926056] amdgpu 0000:00:08.0: amdgpu: reset sdma queue (0:0:0) [ 494.040747] amdgpu 0000:00:08.0: [drm:amdgpu_ring_test_helper [amdgpu]] *ERROR* ring sdma0 test failed (-110) [ 494.048894] amdgpu 0000:00:08.0: amdgpu: Ring sdma0 reset failed [ 494.053713] amdgpu 0000:00:08.0: amdgpu: GPU reset begin!. Source: 1 [ 494.057615] amdgpu 0000:00:08.0: amdgpu: Failed to evict queue 4 [ 494.061866] amdgpu 0000:00:08.0: amdgpu: remove_all_kfd_queues_mes: Failed to remove queue 3 for dev 27203 [ 496.442228] amdgpu 0000:00:08.0: amdgpu: MODE1 reset [ 496.442331] amdgpu 0000:00:08.0: amdgpu: GPU mode1 reset [ 496.448286] amdgpu 0000:00:08.0: amdgpu: GPU smu mode1 reset [ 497.470007] amdgpu 0000:00:08.0: amdgpu: GPU reset succeeded, trying to resume [ 497.485068] amdgpu 0000:00:08.0: amdgpu: PCIE GART of 512M enabled (table at 0x00000087D6B00000). [ 497.487649] amdgpu 0000:00:08.0: amdgpu: VRAM is lost due to GPU reset! [ 497.487657] amdgpu 0000:00:08.0: amdgpu: PSP is resuming... [ 499.368881] amdgpu 0000:00:08.0: amdgpu: GECC is disabled, set amdgpu_ras_enable=1 to enable GECC in next boot cycle if needed [ 499.462332] amdgpu 0000:00:08.0: amdgpu: RAP: optional rap ta ucode is not available [ 499.462336] amdgpu 0000:00:08.0: amdgpu: SECUREDISPLAY: optional securedisplay ta ucode is not available [ 499.462339] amdgpu 0000:00:08.0: amdgpu: SMU is resuming... [ 499.462575] amdgpu 0000:00:08.0: amdgpu: smu driver if version = 0x0000002e, smu fw if version = 0x00000032, smu fw program = 0, smu fw version = 0x00684b00 (104.75.0) [ 499.462581] amdgpu 0000:00:08.0: amdgpu: SMU driver if version not matched [ 499.802760] amdgpu 0000:00:08.0: amdgpu: SMU is resumed successfully! [ 499.815841] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0x4000000 [ 499.816019] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0xc000000 [ 500.124592] amdgpu 0000:00:08.0: amdgpu: [drm] DMUB hardware initialized: version=0x0A000601 [ 500.217191] amdgpu 0000:00:08.0: amdgpu: ring gfx_0.0.0 uses VM inv eng 0 on hub 0 [ 500.217196] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0 [ 500.217198] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0 [ 500.217200] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.1 uses VM inv eng 6 on hub 0 [ 500.217202] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.1 uses VM inv eng 7 on hub 0 [ 500.217204] amdgpu 0000:00:08.0: amdgpu: ring sdma0 uses VM inv eng 8 on hub 0 [ 500.217205] amdgpu 0000:00:08.0: amdgpu: ring sdma1 uses VM inv eng 9 on hub 0 [ 500.217207] amdgpu 0000:00:08.0: amdgpu: ring vcn_unified_0 uses VM inv eng 0 on hub 8 [ 500.217209] amdgpu 0000:00:08.0: amdgpu: ring jpeg_dec uses VM inv eng 1 on hub 8 [ 500.307622] amdgpu 0000:00:08.0: amdgpu: GPU reset(2) succeeded! [ 500.307640] amdgpu 0000:00:08.0: [drm] device wedged, but recovered through reset [ 500.854530] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 501.358512] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 501.862518] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 502.366514] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 502.870484] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 503.374501] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 503.878494] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 504.382487] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 504.886484] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 505.390476] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 505.894449] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 506.398443] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 506.902457] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 507.406452] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 507.910453] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 508.414440] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 508.918432] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 509.422428] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 509.926419] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 510.430415] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 510.934409] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 511.438409] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 511.942398] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 512.446395] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 512.950387] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 513.454382] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 513.958356] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 514.462405] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 514.966360] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 515.470356] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 515.974330] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 516.478344] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 516.982337] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 517.486445] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 517.990326] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 518.494431] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 518.998313] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 519.502422] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 520.006313] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 520.510424] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 521.014306] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 521.518299] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 522.022292] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 538.982269] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 539.486413] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 539.486461] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma1 [ 539.990330] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 539.998341] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 540.502371] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 540.510359] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 541.014385] amdgpu 0000:00:09.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 541.102524] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 541.662447] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 667.539282] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 668.075280] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 668.579334] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 669.083447] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 669.587330] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 670.091344] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 670.596332] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 671.099338] amdgpu 0000:00:08.0: amdgpu: Fence fallback timer expired on ring sdma0 [ 673.147384] amdgpu 0000:00:08.0: amdgpu: Dumping IP State [ 673.244016] amdgpu 0000:00:08.0: amdgpu: Dumping IP State Completed [ 673.244090] amdgpu 0000:00:08.0: amdgpu: [drm] AMDGPU device coredump file has been created [ 673.244093] amdgpu 0000:00:08.0: amdgpu: [drm] Check your /sys/class/drm/card1/device/devcoredump/data [ 673.244096] amdgpu 0000:00:08.0: amdgpu: ring sdma0 timeout, signaled seq=362, emitted seq=366 [ 673.250720] amdgpu 0000:00:08.0: amdgpu: Starting sdma0 ring reset [ 673.259848] amdgpu 0000:00:08.0: amdgpu: reset sdma queue (0:0:0) [ 673.375151] amdgpu 0000:00:08.0: [drm:amdgpu_ring_test_helper [amdgpu]] *ERROR* ring sdma0 test failed (-110) [ 673.383234] amdgpu 0000:00:08.0: amdgpu: Ring sdma0 reset failed [ 673.387916] amdgpu 0000:00:08.0: amdgpu: GPU reset begin!. Source: 1 [ 673.394421] amdgpu 0000:00:08.0: amdgpu: Failed to evict queue 4 [ 673.399916] amdgpu 0000:00:08.0: amdgpu: remove_all_kfd_queues_mes: Failed to remove queue 3 for dev 27203 [ 675.780914] amdgpu 0000:00:08.0: amdgpu: MODE1 reset [ 675.781012] amdgpu 0000:00:08.0: amdgpu: GPU mode1 reset [ 675.788320] amdgpu 0000:00:08.0: amdgpu: GPU smu mode1 reset [ 676.803585] amdgpu 0000:00:08.0: amdgpu: GPU reset succeeded, trying to resume [ 676.818582] amdgpu 0000:00:08.0: amdgpu: PCIE GART of 512M enabled (table at 0x00000087D6B00000). [ 676.821090] amdgpu 0000:00:08.0: amdgpu: VRAM is lost due to GPU reset! [ 676.821095] amdgpu 0000:00:08.0: amdgpu: PSP is resuming... [ 678.701332] amdgpu 0000:00:08.0: amdgpu: GECC is disabled, set amdgpu_ras_enable=1 to enable GECC in next boot cycle if needed [ 678.794924] amdgpu 0000:00:08.0: amdgpu: RAP: optional rap ta ucode is not available [ 678.794928] amdgpu 0000:00:08.0: amdgpu: SECUREDISPLAY: optional securedisplay ta ucode is not available [ 678.794934] amdgpu 0000:00:08.0: amdgpu: SMU is resuming... [ 678.795135] amdgpu 0000:00:08.0: amdgpu: smu driver if version = 0x0000002e, smu fw if version = 0x00000032, smu fw program = 0, smu fw version = 0x00684b00 (104.75.0) [ 678.795140] amdgpu 0000:00:08.0: amdgpu: SMU driver if version not matched [ 679.133686] amdgpu 0000:00:08.0: amdgpu: SMU is resumed successfully! [ 679.146767] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0x4000000 [ 679.146944] amdgpu 0000:00:08.0: amdgpu: program CP_MES_CNTL : 0xc000000 [ 679.452608] amdgpu 0000:00:08.0: amdgpu: [drm] DMUB hardware initialized: version=0x0A000601 [ 679.543931] amdgpu 0000:00:08.0: amdgpu: ring gfx_0.0.0 uses VM inv eng 0 on hub 0 [ 679.543937] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0 [ 679.543939] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0 [ 679.543941] amdgpu 0000:00:08.0: amdgpu: ring comp_1.0.1 uses VM inv eng 6 on hub 0 [ 679.543942] amdgpu 0000:00:08.0: amdgpu: ring comp_1.1.1 uses VM inv eng 7 on hub 0 [ 679.543944] amdgpu 0000:00:08.0: amdgpu: ring sdma0 uses VM inv eng 8 on hub 0 [ 679.543946] amdgpu 0000:00:08.0: amdgpu: ring sdma1 uses VM inv eng 9 on hub 0 [ 679.543948] amdgpu 0000:00:08.0: amdgpu: ring vcn_unified_0 uses VM inv eng 0 on hub 8 [ 679.543950] amdgpu 0000:00:08.0: amdgpu: ring jpeg_dec uses VM inv eng 1 on hub 8 [ 862.924362] Workqueue: amdgpu-reset-dev drm_sched_job_timedout [amd_sched] [ 862.924423] dma_fence_default_wait+0x1e1/0x220 [ 862.924432] ? __pfx_dma_fence_default_wait_cb+0x10/0x10 [ 862.924438] dma_fence_wait_timeout+0x116/0x140 [ 862.924444] amdgpu_fence_emit+0x197/0x200 [amdgpu] [ 862.925240] amdgpu_ib_schedule+0x4a1/0x900 [amdgpu] [ 862.925969] sdma_v7_0_ring_test_ib+0x145/0x260 [amdgpu] [ 862.926854] amdgpu_ib_ring_tests+0x14b/0x1d0 [amdgpu] [ 862.927676] amdgpu_device_reinit_after_reset+0xf0/0x360 [amdgpu] [ 862.928476] amdgpu_do_asic_reset.part.0+0x56/0x1b0 [amdgpu] [ 862.929249] amdgpu_device_asic_reset+0x454/0x6b0 [amdgpu] [ 862.929944] ? amdgpu_device_halt_activities.isra.0+0x1a8/0x280 [amdgpu] [ 862.930730] amdgpu_device_gpu_recover+0x24f/0x400 [amdgpu] [ 862.931533] amdgpu_job_timedout+0x242/0x630 [amdgpu] [ 862.947651] Workqueue: kfd_process_wq kfd_process_wq_release [amdgpu] [ 862.948661] kfd_process_wq_release+0x5a/0x480 [amdgpu] [ 862.965469] dma_fence_default_wait+0x1e1/0x220 [ 862.965474] ? __pfx_dma_fence_default_wait_cb+0x10/0x10 [ 862.965480] dma_fence_wait_timeout+0x116/0x140 [ 862.981598] dma_fence_default_wait+0x1e1/0x220 [ 862.981603] ? __pfx_dma_fence_default_wait_cb+0x10/0x10 [ 862.981609] dma_fence_wait_timeout+0x116/0x140 [ 862.997473] kfd_create_process+0x6e/0xd30 [amdgpu] [ 862.998450] kfd_open+0x34/0x130 [amdgpu]