@ph7

I wasnt sure who they were addressing you or me.

Any way. I have updated once again to latest master commit 372a2. All seems to be working again.... Rebooted XO just to be sure.

@ph7

Well prior to installing the latest commit from today will was back working. Just updated to 28c5e and now back to 1 vm restore point again.

Not sure why these multiple issues keep popping up and going away just to pop back up again between releases.

I still have two open support tickets open waiting for responses for work. One about the recent 8.3 updates other about mirror backup broken.

@florent @danp

At work I am seeing this error as well... I have opened a support ticket Ticket#7755762

I posted in the original thread that this issue only exists in this test branch. When i switched back to master branch all restore points were visable.

Edit - Just updated to 875ce and now back to single vm listed for restore... Commit bfb0a05 did not have issue.

Edit 2 - Updated to a6c50 and restore points are back again.

@MathieuRA said:

@ph7 @acebmxer
Hi, regarding your backups which do not appear on the XO5 restore page, I suggest you to open a new topic so that this one remains focused on XO6 dashboards

Once I updated back to master are restore points are visible again. Issue only appears with this test branch.

@ph7 @mathieura

Whats even more strange... I just updated back to master branch and now all v6 dashboards are working sorta.... Initially all spinning circles from pool to every vm. But If i double click on the vm the dashboard loads. Same for Pool. This was not the case previously. Before i could double click or click refresh all day long never load.

I have noticed one other thing on this test branch.... All my restore points for vms are missing except for 1 vm....

2026-03-31T17_19_44.804Z - backup NG.txt

@mathieura

I have made no changes since applying your test branch... Now all vms have a working dashboard....

and pool dashboard...

@MathieuRA

yeah sorry about that... Working off laptop through multiple vms

Saw your last post. let me know if you need more info from me. Will wait for next update.

@MathieuRA

Just finish updating to your test branch... The pool dashboard loads ok but vm backup info still infinite loop

@MathieuRA said:

@ph7 I am talking about that:

The result is what you can see in the response tab

from my install...

.pool-dashboard-cpu-provisioning .total[data-v-68cd38fd] {
    display: grid;
    grid-template-columns: 1fr 1fr;
    margin-block-start:auto}

.vts-progress-bar-group[data-v-c6e74eac] {
    display: flex;
    flex-direction: column;
    gap: 1.6rem
}

.ui-card-subtitle[data-v-72bac4cd] {
    color: var(--color-brand-txt-base);
    border-bottom: .1rem solid var(--color-brand-txt-base);
    display: flex;
    justify-content: space-between;
    gap: .8rem
}

.pool-dashboard-patches[data-v-c2490c67] {
    max-height: 46.2rem
}

.pool-dashboard-patches .missing-patches-info[data-v-c2490c67] {
    color: var(--color-danger-txt-base)
}

.pool-dashboard-patches .table-wrapper[data-v-c2490c67] {
    overflow-y: auto;
    margin-inline:-2.4rem;margin-block-end:-1.2rem;border-block:.1rem solid var(--color-neutral-border)}

.pool-dashboard-patches .table-wrapper .table[data-v-c2490c67] {
    margin-top: -.1rem
}

.pool-storages-usage .total[data-v-74099ef4] {
    display: grid;
    grid-template-columns: 1fr 1fr;
    margin-block-start:auto}

.dashboard[data-v-db8e7621] {
    margin: .8rem
}

.dashboard .row[data-v-db8e7621] {
    display: grid;
    gap: .8rem
}

.dashboard .row+.row[data-v-db8e7621] {
    margin-top: .8rem
}

.dashboard .first-row[data-v-db8e7621] {
    grid-template-columns: minmax(10rem,1fr) minmax(20rem,2fr) minmax(10rem,1fr);
    grid-template-areas: "status alarms patches"
}

.dashboard .status[data-v-db8e7621] {
    grid-area: status
}

.dashboard .alarms[data-v-db8e7621] {
    grid-area: alarms;
    height: 46.2rem
}

.dashboard .patches[data-v-db8e7621] {
    grid-area: patches
}

.dashboard .second-row[data-v-db8e7621] {
    grid-template-columns: repeat(3,minmax(20rem,1fr));
    grid-template-areas: "first-column second-column third-column"
}

.dashboard .column[data-v-db8e7621] {
    display: flex;
    flex-direction: column;
    gap: .8rem
}

.dashboard .first-column[data-v-db8e7621] {
    grid-area: first-column
}

.dashboard .second-column[data-v-db8e7621] {
    grid-area: second-column
}

.dashboard .third-column[data-v-db8e7621] {
    grid-area: third-column
}

.dashboard.mobile[data-v-db8e7621] {
    display: flex;
    flex-direction: column;
    gap: .8rem
}

.dashboard.mobile .row[data-v-db8e7621] {
    display: flex;
    flex-direction: column;
    gap: .8rem;
    margin-top: 0
}

@MathieuRA said:

@acebmxer so the endpoint seems to respond correctly.

Can you confirm that the pool dashboard UI is still displaying infinite loaders?

yes still loading circles...

I will be away from my computer for awhile so i wont be able to check. Will post when i can.

@MathieuRA

From pool dashboard...

Pool dashboard the same also...

@ph7 @mathieura
feels like i am sliding down hill.....

on latest commit 40179

did reboot xo after update.

Edit - if i refresh the browser the "Last 3 backup Archives" and "last replication" stop spinning. Also tried in incognito mode to eliminate cookie issues.

sudo journalctl -u xo-server -n 75
Mar 31 10:17:46 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:17:46 xo-ce xo-server[852]: }
Mar 31 10:17:46 xo-ce xo-server[852]: 2026-03-31T10:17:46.484Z xo:rest-api:error-handler INFO [GET] /vms/fb72a8d7-a039-849f-b547-24fc56f056ba/dashbo>
Mar 31 10:17:48 xo-ce xo-server[852]: 2026-03-31T10:17:48.391Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:17:48 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:17:48 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:17:48 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:17:48 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:17:48 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:17:48 xo-ce xo-server[852]: }
Mar 31 10:17:48 xo-ce xo-server[852]: 2026-03-31T10:17:48.391Z xo:rest-api:error-handler INFO [GET] /vms/138538a8-ef52-4d0a-4433-5ebb31d7e152/dashbo>
Mar 31 10:17:57 xo-ce xo-server[852]: 2026-03-31T10:17:57.455Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:17:57 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:17:57 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:17:57 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:17:57 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:17:57 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:17:57 xo-ce xo-server[852]: }
Mar 31 10:17:57 xo-ce xo-server[852]: 2026-03-31T10:17:57.455Z xo:rest-api:error-handler INFO [GET] /vms/b46151e7-690c-9513-126e-3824da2b0ca9/dashbo>
Mar 31 10:18:19 xo-ce xo-server[852]: 2026-03-31T10:18:19.646Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:19 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:19 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:19 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:19 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:19 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:19 xo-ce xo-server[852]: }
Mar 31 10:18:19 xo-ce xo-server[852]: 2026-03-31T10:18:19.647Z xo:rest-api:error-handler INFO [GET] /vms/24714e97-146f-76f2-699c-bc93116b6a90/dashbo>
Mar 31 10:18:34 xo-ce xo-server[852]: 2026-03-31T10:18:34.797Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:34 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:34 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:34 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:34 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:34 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:34 xo-ce xo-server[852]: }
Mar 31 10:18:34 xo-ce xo-server[852]: 2026-03-31T10:18:34.797Z xo:rest-api:error-handler INFO [GET] /vms/fb72a8d7-a039-849f-b547-24fc56f056ba/dashbo>
Mar 31 10:18:39 xo-ce xo-server[852]: 2026-03-31T10:18:39.691Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:39 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:39 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:39 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:39 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:39 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:39 xo-ce xo-server[852]: }
Mar 31 10:18:39 xo-ce xo-server[852]: 2026-03-31T10:18:39.692Z xo:rest-api:error-handler INFO [GET] /vms/fb72a8d7-a039-849f-b547-24fc56f056ba/dashbo>
Mar 31 10:18:40 xo-ce xo-server[852]: 2026-03-31T10:18:40.921Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:40 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:40 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:40 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:40 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:40 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:40 xo-ce xo-server[852]: }
Mar 31 10:18:40 xo-ce xo-server[852]: 2026-03-31T10:18:40.921Z xo:rest-api:error-handler INFO [GET] /vms/fb72a8d7-a039-849f-b547-24fc56f056ba/dashbo>
Mar 31 10:18:42 xo-ce xo-server[852]: 2026-03-31T10:18:42.439Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:42 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:42 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:42 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:42 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:42 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:42 xo-ce xo-server[852]: }
Mar 31 10:18:42 xo-ce xo-server[852]: 2026-03-31T10:18:42.439Z xo:rest-api:error-handler INFO [GET] /vms/138538a8-ef52-4d0a-4433-5ebb31d7e152/dashbo>
Mar 31 10:18:49 xo-ce xo-server[852]: 2026-03-31T10:18:49.547Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:49 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:49 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:49 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:49 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:49 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:49 xo-ce xo-server[852]: }
Mar 31 10:18:49 xo-ce xo-server[852]: 2026-03-31T10:18:49.547Z xo:rest-api:error-handler INFO [GET] /vms/b46151e7-690c-9513-126e-3824da2b0ca9/dashbo>
Mar 31 10:18:51 xo-ce xo-server[852]: 2026-03-31T10:18:51.696Z xo:rest-api:error-handler ERROR Cannot read properties of undefined (reading 'status'>
Mar 31 10:18:51 xo-ce xo-server[852]:   error: TypeError: Cannot read properties of undefined (reading 'status')
Mar 31 10:18:51 xo-ce xo-server[852]:       at file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:240:36
Mar 31 10:18:51 xo-ce xo-server[852]:       at Array.map (<anonymous>)
Mar 31 10:18:51 xo-ce xo-server[852]:       at VmService.#getBackupsInfo (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/vms/vm.service.mjs:>
Mar 31 10:18:51 xo-ce xo-server[852]:       at promiseWriteInStream (file:///opt/xen-orchestra/@xen-orchestra/rest-api/dist/helpers/utils.helper.mjs>
Mar 31 10:18:51 xo-ce xo-server[852]: }
Mar 31 10:18:51 xo-ce xo-server[852]: 2026-03-31T10:18:51.696Z xo:rest-api:error-handler INFO [GET] /vms/24714e97-146f-76f2-699c-bc93116b6a90/dashbo>
lines 36-75/75 (END)

@MathieuRA

I am guessing i did thoroughly check out your fix against all vms.... Some vms will populate others will not... Here are more of the log info. Let me know if you still need more if this is caused by my script.

xo-logs.txt

I ran this all through claude code and this is its thoughts...

The server logs don't show anything related to the backup spinner, but there are two issues worth addressing:

1. Invalid environment assignment (line 11 of the service file)


/etc/systemd/system/xo-server.service:11: Invalid environment assignment, ignoring
Something in the service file's Environment= or EnvironmentFile= line is malformed. This could be from the install script. Worth checking what's on line 11 of that file.

2. PAM limits failures (recurring)


pam_limits: Could not set limit for 'nofile' to soft=1024, hard=1048576
pam_limits: Could not set limit for 'memlock' to soft=775520256, hard=...
The xo user can't set its own resource limits when running sudo. This suggests the systemd LimitNOFILE/LimitMEMLOCK directives in the service file conflict with what PAM is trying to apply.

3. listMissingPatches / updater plugin busy — repeated but harmless; XO polls for host patches and races with the xcp-ng updater. Not related to your issue.

4. WebSocket ECONNRESET/UND_ERR_SOCKET — normal; long-lived connections to xcp-ng hosts drop and reconnect.

None of these explain the backup spinner — that's still a UI/API retrieval bug. But the service file issues are worth fixing. Can you share the output of:


cat -n /etc/systemd/system/xo-server.service
That'll show exactly what's wrong on line 11, and I can check if the install script is generating it incorrectly.

cat -n /etc/systemd/system/xo-server.service

     1  [Unit]
     2  Description=Xen Orchestra Server
     3  After=network-online.target redis.service
     4  Wants=network-online.target
     5
     6  [Service]
     7  Type=simple
     8  User=xo-service
     9
    10  Environment="NODE_ENV=production"
    11  WorkingDirectory=/opt/xen-orchestra/packages/xo-server
    12  ExecStartPre=/bin/mkdir -p /run/xo-server/mounts
    13  ExecStartPre=/bin/chmod 755 /run/xo-server/mounts
    14  ExecStart=/usr/bin/node /opt/xen-orchestra/packages/xo-server/dist/cli.mjs
    15  Restart=always
    16  RestartSec=10
    17  SyslogIdentifier=xo-server
    18
    19  # Runtime directory
    20  RuntimeDirectory=xo-server
    21  RuntimeDirectoryMode=0755
    22
    23  # Allow binding to privileged ports (80/443)
    24  AmbientCapabilities=CAP_NET_BIND_SERVICE
    25  # Bounding set: ceiling for all processes in this service tree.
    26  # CAP_NET_BIND_SERVICE: bind to ports 80/443
    27  # CAP_SETUID/CAP_SETGID/CAP_AUDIT_WRITE: required for sudo to function
    28  # CAP_SYS_ADMIN: required for mount syscall (NFS/CIFS remotes)
    29  CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_SETUID CAP_SETGID CAP_SYS_ADMIN CAP_AUDIT_WRITE
    30
    31  [Install]
    32  WantedBy=multi-user.target

The current service file looks completely fine — line 11 is WorkingDirectory=... which is valid. The "Invalid environment assignment" warning in the Feb 24 logs was from an older version of the file that has since been fixed (likely by a re-run of your install script).

The current file has no issues. All directives are correct:

Environment="NODE_ENV=production" — valid syntax
CapabilityBoundingSet — looks correct from the recent commit fixes
Everything else is standard
The PAM limits errors you saw are also unrelated to the service file syntax — those happen because pam_limits tries to apply /etc/security/limits.conf rules when xo-server runs sudo, but systemd's own LimitNOFILE takes precedence and PAM can't override it. This is expected behavior and harmless.

Bottom line: The server logs don't reveal any cause for the backup spinner. The spinner is almost certainly this known XO behavior: when the UI subscribes to backup log data for a VM, if the response takes too long or returns in an unexpected format, the loading state never resolves. It's a frontend bug, not a server configuration issue.

@MathieuRA

Dont think you pushed this already unless i am mistaken. But on Commit 9226c

One vm populates the backup second on dashboard while the other do not. this happens to be the same vm check to confirm your changes. I dont recall if i checked other vms.

But on Commit 9226c 1 vm dashboard loads correctly.

@MathieuRA

That seemed to work....

Edit - Updated back to master commit. and broken again as expected... So you patched branch works...

@AlexanderK
The inter-active menu, The menu will show the running script commit against main commit along with showing Xen Orchestra installed commit vs master commit. The script should automatically self-update to latest version.

Just to name a few...