We ran into the same issue as well. We removed ALL snapshots of this vm and then it started working again. Comes at a cost since we had to remove the non-backup-related snapshots as well...

@olivierlambert Yes - we have compiled it from sources for the lab tests.

@olivierlambert We have been using the master channel. We pulled two files about two weeks ago.

Hi everyone,

we are testing an xcp-ng setup to hopefully replace a vmware system soon. We tested with delta backups and an actual set of vms with about 3-5 TB storage.

We noticed that backups get stuck on one vm every once in a while. The situation resolves most of the times as soon as we restart the xoa vm. Being stuck, new snapshots for backups don't even start. The status is only "orange" in xoa.

The error log of the xoa shows the following:

Mar  3 09:26:50 ktmr-vm-xoa xo-server[20439]: 2023-03-03T08:26:50.509Z xo:backups:mergeWorker INFO merge in progress {
Mar  3 09:26:50 ktmr-vm-xoa xo-server[20439]:   done: 68,
Mar  3 09:26:50 ktmr-vm-xoa xo-server[20439]:   parent: 'xo-vm-backups/22676691-e76d-58fa-e73d-0cc9ec0d7150/vdis/4c1f1db8-b70f-4e1c-aca7-74b457edacb2/0793f530-5
2f5-4d6f-8bb8-5710117e8eb3/20230227T180042Z.vhd',
Mar  3 09:26:50 ktmr-vm-xoa xo-server[20439]:   progress: 0,
Mar  3 09:26:50 ktmr-vm-xoa xo-server[20439]:   total: 21554
Mar  3 09:26:50 ktmr-vm-xoa xo-server[20439]: }

The messages has been repeating itself for about 12 hours now. The progress remains 0 and the done at 68.

Did anyone notice anything similar, yet? Any suggestions how to resolve this?

Thanks,
Hannes

@florent @olivierlambert
Thank you both for your quick reaction! I know that you guys have a lot on your hands with this great piece of software!

Just let me know as soon as we can test-drive something.

@olivierlambert said in Restore Delta Backup with RAM:

Ideally, we could raise the priority if it will be related to a potential future revenue

I am hoping the same but cannot promise anything, yet On the other side there might some people using the memory-enabled feature not knowing that it's not working as expected which is a win, too

@olivierlambert Thanks a lot for your quick reply!

Someone will take a look here, trying to reproduce the issue. Can you confirm you have it with XOA on latest release channel? (ping @marcungeschikts so we can assign someone internally)

XOA is on the latest release channel.

In the meantime : in theory, the PV drivers inside your VMs are aware about the snapshot. Have you ever tested and see if it corrupted anything?

I have tested it and never ended up with a corrupt db. On the other side we are talking about MS SQL and Oracle DB servers running in production so it really needs to be safe.

Hi everyone,

I have a recent install of xcp-ng 8.2 and a Xen Orchestra version of 5.78.0 running. I am doing the following backup scheme:
Method: Delta Backup
Snapshot Mode: With Memory
VM: Debian Buster with management agent installed / Win10 Pro with MA.

When I am restoring the VM the VM is restored in an halted but not suspended state which "leaves the ram behind". Since we plan to backup a SQL server it would be great to have the ram as well for consistency.

Interesting is that during the restore process the main VDI and the "VDI Suspend Image" get imported. At the end the machine is halted though and when I start the machine it does a regular boot.

Investigating this issue I found an old Github (see https://github.com/vatesfr/xen-orchestra/issues/5061) issue from 2020 which still seems to be open.

Does anyone have any info on this? I would like to pitch XOA at work for production systems as well but we need this feature

toni-1 created this issue in vatesfr/xen-orchestra

closed Restored VM from RAM enabled backup is not in suspend mode #5061

We seem to experience the same issues. We are on commit d3071. It seems that xoa mislables the backups as well.
Even though we enforced a full backup through the schedule option, it is labled as a delta in the restore process menu.
The xoa backup "log" in the interface shows full as well instead of delta and the amount of data transferred is clearly a full.