Continuos replication failing with "checkBaseVdis() failed"
-
@pedro_udifar What is your current commit for the XO VM? Can you share the full error log as an attachment?
-
@Danp Hi!
Running Xen Orchestra, commit 44e5b (19 commits behind).
Here is a log
deltalog.txtThank you!
-
@pedro_udifar Please update to the latest code base and then let us know if you still encounter this issue.
-
Hi @Danp ,
I've updated XO to the latest commit and the error is still happening.
-
Hello,
More info on this. I think this error is related with the recent "Distribute replication accross the storage repositories" option that I turned on. Before this option existed, I was splitting the VMs into groups, with tags, and using several jobs with "smart mode" to send each group to a specific SR on the destination replication pool. With this new option I didn't have to do this anymore, it would simplify my replication job.
That said, I still have a group that I want to replicate to a local host disk. This group doesn't fail with "checkBaseVdis() failed" but I do get duplicated VMs on the destination pool, even though the job's schedule is set to only 1 replications retention. Is XO failing to delete the previous VM replication ?
(The same VM duplicated on the replicated pool even though replication retention is set to 1)
(Job / schedule retention settings)....So I have to prune the duplicated VMs to make sure I don't run out of space on the target pool repository.
I think I'll set up a new job with a single test VM so I can run it several times and try to gather more info.
-
Hello!
So I created a new job and a small test vm to gather info. I did 3 replications runs and the last 2 failed. Backups are working thoug.
Since this would be a large post, I created a pdf with the test cycle.
...
Only to find out that those are not allowed on this forum! So I captured the pdf pages...
And the logs from the jobs:
{ "data": { "mode": "delta", "reportWhen": "always", "backupReportTpl": "mjml" }, "id": "1784293017099", "jobId": "10d0331c-694e-4681-8860-33fef3790552", "jobName": "TESTCRANDBK", "message": "backup", "scheduleId": "bc7b29ce-25b2-4c66-9186-b1d4c314b979", "start": 1784293017099, "status": "success", "tasks": [ { "id": "0mroxztk5-9yuipq34src", "start": 1784293020005, "status": "success", "tasks": [ { "id": "0mroxztkj-izqz7v0ufn", "start": 1784293020019, "status": "success", "end": 1784293020027, "result": { "merge": false, "size": 0 }, "message": "clean-vm" }, { "id": "0mroxztz1-v8ycdmtgo9k", "start": 1784293020541, "status": "success", "end": 1784293036537, "result": "20683621-7819-7661-91a2-93c377a7345a", "message": "snapshot" }, { "id": "0mroy06bf-9frdqsgnpym", "start": 1784293036539, "status": "success", "tasks": [ { "id": "0mroy0aun-bscsuoyzrpu", "start": 1784293042415, "status": "success", "tasks": [ { "id": "0mroy3r14-2sk7dgyqrbl", "start": 1784293203352, "status": "success", "end": 1784293207572, "message": "target snapshot" } ], "end": 1784293207573, "result": { "size": 9254731776 }, "message": "transfer" } ], "end": 1784293207664, "message": "export", "data": { "id": "a729e916-4cd8-2208-7dce-329e6b4af892", "isFull": true, "name_label": "xxxxx", "type": "SR" } }, { "id": "0mroy06bg-eyd2s4tidgg", "start": 1784293036540, "status": "success", "end": 1784293207664, "message": "export", "data": { "id": "b0b77301-4625-a13b-7022-9512f927944a", "isFull": true, "name_label": "xxxxxx", "type": "SR" } }, { "id": "0mroy06bh-ra3dxpusxr8", "start": 1784293036541, "status": "success", "end": 1784293207664, "message": "export", "data": { "id": "7054c407-d620-2c9f-5810-1e192a04ffe2", "isFull": true, "name_label": "xxxxxxxxx", "type": "SR" } }, { "id": "0mroy06bi-tdyzypsg58", "start": 1784293036542, "status": "success", "end": 1784293207664, "message": "export", "data": { "id": "621c9b87-4f58-987b-2c07-e8dee67da73a", "isFull": true, "name_label": "xxxxxxxxxx", "type": "SR" } }, { "id": "0mroy06be-u3p9nky3uwd", "start": 1784293036538, "status": "success", "tasks": [ { "id": "0mroy0aus-ax44kkpim1", "start": 1784293042420, "status": "success", "end": 1784293200916, "result": { "size": 9257013760 }, "message": "transfer" }, { "id": "0mroy434q-md07n6vujw", "start": 1784293219034, "status": "success", "end": 1784293219099, "result": { "merge": false, "size": 0 }, "message": "clean-vm" } ], "end": 1784293219102, "message": "export", "data": { "id": "c1fdab1d-5535-42e4-b282-1a0282df1081", "isFull": true, "type": "remote" } } ], "infos": [ { "message": "Transfer data using NBD" }, { "message": "will delete snapshot data" }, { "data": { "vdiRef": "OpaqueRef:080d2018-c2a3-aa6f-89a1-b3eaf72c3de5" }, "message": "Snapshot data has been deleted" } ], "end": 1784293219102, "message": "backup VM", "data": { "id": "d771731a-f132-7365-be50-71f19424f96e", "type": "VM", "name_label": "TESTREPL", "progress": 0 } } ], "end": 1784293219104, "infos": [ { "data": { "vms": [ "d771731a-f132-7365-be50-71f19424f96e" ] }, "message": "vms" } ] } { "data": { "mode": "delta", "reportWhen": "always", "backupReportTpl": "mjml" }, "id": "1784293990343", "jobId": "10d0331c-694e-4681-8860-33fef3790552", "jobName": "TESTCRANDBK", "message": "backup", "scheduleId": "bc7b29ce-25b2-4c66-9186-b1d4c314b979", "start": 1784293990343, "status": "success", "tasks": [ { "id": "0mroykonr-nktgebhtb4k", "start": 1784293993431, "status": "success", "tasks": [ { "id": "0mroykoo5-171n48gwq3r", "start": 1784293993445, "status": "success", "end": 1784293993493, "result": { "merge": false, "size": 0 }, "message": "clean-vm" }, { "id": "0mroykp31-4r5wrf9blru", "start": 1784293993981, "status": "success", "end": 1784294007573, "result": "9ff54ec5-0247-4692-945b-c183206ae6dd", "message": "snapshot" }, { "id": "0mroykzkm-ykzk53um8l", "start": 1784294007574, "status": "success", "tasks": [ { "id": "0mroyl3qn-vzompxsw79o", "start": 1784294012975, "status": "success", "end": 1784294018591, "result": { "size": 396481024 }, "message": "transfer" }, { "id": "0mroyljgo-nlch68rkbil", "start": 1784294033352, "status": "success", "end": 1784294033444, "result": { "merge": false, "size": 0 }, "message": "clean-vm" } ], "end": 1784294033446, "message": "export", "data": { "id": "c1fdab1d-5535-42e4-b282-1a0282df1081", "isFull": false, "type": "remote" } } ], "warnings": [ { "message": "the writer AggregatedIncrementalXapiWriter has failed the step writer.checkBaseVdis() with error Cannot read properties of undefined (reading 'get'). It won't be used anymore in this job execution." } ], "infos": [ { "message": "Transfer data using NBD" }, { "message": "will delete snapshot data" }, { "data": { "vdiRef": "OpaqueRef:77605638-5e58-4a10-c2de-bc6e77f1ad61" }, "message": "Snapshot data has been deleted" } ], "end": 1784294033447, "message": "backup VM", "data": { "id": "d771731a-f132-7365-be50-71f19424f96e", "type": "VM", "name_label": "TESTREPL", "progress": 0 } } ], "end": 1784294033449, "infos": [ { "data": { "vms": [ "d771731a-f132-7365-be50-71f19424f96e" ] }, "message": "vms" } ] } { "data": { "mode": "delta", "reportWhen": "always", "backupReportTpl": "mjml" }, "id": "1784295416670", "jobId": "10d0331c-694e-4681-8860-33fef3790552", "jobName": "TESTCRANDBK", "message": "backup", "scheduleId": "bc7b29ce-25b2-4c66-9186-b1d4c314b979", "start": 1784295416670, "status": "success", "tasks": [ { "id": "0mrozf8yr-lix8jxbm6pd", "start": 1784295419428, "status": "success", "tasks": [ { "id": "0mrozf8zg-zddbxm901mc", "start": 1784295419452, "status": "success", "end": 1784295419537, "result": { "merge": false, "size": 0 }, "message": "clean-vm" }, { "id": "0mrozf9dv-7nj082i92kp", "start": 1784295419971, "status": "success", "end": 1784295426890, "result": "18312e82-adb6-2dc5-b13e-2fd9a4b2bf77", "message": "snapshot" }, { "id": "0mrozfeq3-w3y5xrwtqse", "start": 1784295426891, "status": "success", "tasks": [ { "id": "0mrozfrvy-4g1gefr14hx", "start": 1784295443950, "status": "success", "end": 1784295445177, "result": { "size": 92319744 }, "message": "transfer" }, { "id": "0mrozfw5f-mx0xt5f7wjm", "start": 1784295449475, "status": "success", "end": 1784295449563, "result": { "merge": false, "size": 0 }, "message": "clean-vm" } ], "end": 1784295449565, "message": "export", "data": { "id": "c1fdab1d-5535-42e4-b282-1a0282df1081", "isFull": false, "type": "remote" } } ], "warnings": [ { "message": "the writer AggregatedIncrementalXapiWriter has failed the step writer.checkBaseVdis() with error Cannot read properties of undefined (reading 'get'). It won't be used anymore in this job execution." } ], "infos": [ { "message": "Transfer data using NBD" }, { "message": "will delete snapshot data" }, { "data": { "vdiRef": "OpaqueRef:6a833d28-2d2c-5fe6-858b-02d2fd41b919" }, "message": "Snapshot data has been deleted" } ], "end": 1784295449566, "message": "backup VM", "data": { "id": "d771731a-f132-7365-be50-71f19424f96e", "type": "VM", "name_label": "TESTREPL", "progress": 0 } } ], "end": 1784295449567, "infos": [ { "data": { "vms": [ "d771731a-f132-7365-be50-71f19424f96e" ] }, "message": "vms" } ] }Hope it helps
-
Hello!
I realize now that the testing procedure was flawed - I shouldn't have "forced started" the replicated VM. I should have started a copy.
But I don't think it would have changed the end result anyway.Over the weekend I reverted my replications to groups, each group sending to a specific repository.
Now they replicate but I'm getting duplicates.Can it be that "The long term retention of backups" is also affecting the replication retention ?
I want backup to have long term retention and replication to only keep the latest replication.
Cheers,
Pedro -
@pedro_udifar This is indeed a particular case. Can you switch xo-server log to debug please? Then we would need the journalctl from your xo-server. You can give it to me through MP if you prefer.
-
@pierrebrunet Hi!
I'll send you xo logs of the errors.
In the meantime, I've deleted "Long-term retention of backups" settings and I no longer get duplicated VMs on the DR pool.
So it seems that this setting is also being used for replication and I cannot use it. Don't have the space for it on my storage, with thick snapshots on lvmoiscsi.
-
Got this issue since 2026-07-12.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login