Thank you 100x! I'm not sure why but looking though my shell history I did do a
yum install gpumon
Then for some stupid reason I removed it. I think I was experimenting with GPU pass through but decided against it and later removed it.
Reinstalling gpumon fixed it. Thank you so much for your help.
Thanks for the hint.
I see this on a xapi service restart but not sure how to address
Feb 11 05:57:30 xcp-hotdog xapi: [debug||0 |dbsync (update_env) R:e76d7a56e52b|dbsync] Sync: sync_chipset_info
Feb 11 05:57:30 xcp-hotdog xapi: [debug||0 |dbsync (update_env) R:e76d7a56e52b|dbsync] Sync: sync_gpus
Feb 11 05:57:30 xcp-hotdog xapi: [error||0 |dbsync (update_env) R:e76d7a56e52b|xapi] Ignoring error parsing /usr/share/nvidia/vgpu/vgpuConfig.xml: (Sys_error\x0A "/usr/share/nvidia/vgpu/vgpuConfig.xml: No such file or directory")\x0ARaised by primitive operation at file "stdlib.ml", line 398, characters 28-54\x0ACalled from file "stdlib.ml" (inlined), line 403, characters 2-45\x0ACalled from file "ocaml/xapi/xapi_vgpu_type.ml", line 690, characters 15-32\x0A\x0A
Feb 11 05:57:31 xcp-hotdog xapi: [ info||51 UNIX /var/lib/xcp/xapi|session.login_with_password D:d36248a55481|xapi_session] Session.create trackid=670029dd7abd485292112f39d0a6dbb0 pool=false uname= originator=xen-api-scripts-perfmon is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Feb 11 05:57:31 xcp-hotdog xapi: [debug||52 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:93d911ab91aa created by task D:d36248a55481
Feb 11 05:57:31 xcp-hotdog xapi: [debug||57 UNIX /var/lib/xcp/xapi|session_check D:187aaaa66aa3|session_check] Session check failed: missing reference; tbl = session, ref = OpaqueRef:9f5dd4f9-5a0b-4b6b-a373-e196ab1e7ef9
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] session_check D:187aaaa66aa3 failed with exception Server_error(SESSION_INVALID, [ OpaqueRef:9f5dd4f9-5a0b-4b6b-a373-e196ab1e7ef9 ])
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] Raised Server_error(SESSION_INVALID, [ OpaqueRef:9f5dd4f9-5a0b-4b6b-a373-e196ab1e7ef9 ])
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] 1/6 xapi Raised at file ocaml/xapi/session_check.ml, line 63
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] 2/6 xapi Called from file ocaml/xapi/server_helpers.ml, line 100
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] 3/6 xapi Called from file ocaml/xapi/server_helpers.ml, line 121
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] 4/6 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] 5/6 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace] 6/6 xapi Called from file lib/backtrace.ml, line 177
Feb 11 05:57:31 xcp-hotdog xapi: [error||57 UNIX /var/lib/xcp/xapi||backtrace]
Single host and same host same pool. It was not recently added.
My XCP host rebooted and seems to be stuck in dbsync pending.
[18:03 xcp ~]# xe task-list
uuid ( RO) : 9a363d30-a7c5-a8fa-a388-a795acbd52a6
name-label ( RO): dbsync (update_env)
name-description ( RO):
status ( RO): pending
progress ( RO): 0.000
[18:02 xcp ~]# list_domains
id | uuid | state
0 | 4edb1d69-4872-482b-9835-de12e4be929e | R
[18:02 xcp ~]# xe host-list
uuid ( RO) : 5cc7060b-4972-446f-acd3-ef9fe520eb01
name-label ( RW): xcp
name-description ( RW): Default install
[18:02 xcp ~]# xe host-enable uuid=5cc7060b-4972-446f-acd3-ef9fe520eb01
The host toolstack is still initialising. Please wait.
xensource log from restart attachment
xcp.txt
Restarting the tool stack still shows it stuck. I'm not sure what to try next.
Thanks in advance for any suggestions.