Stuck VDI Exports After Latest Upgrade
-
Hello,
I wanted to report that it looks like after the latest upgrade to XO sources a14c4bbb1c528456afff4dbb040d8b0b3cb33742 it looks like there are exporting content of VDI tasks stuck running in he pool tasks. even though the backup finished and was successful. I have attached a screenshot and log from the backup. Please let me know if anymore information is needed.
-
I'm seeing the same thing
XO Source: Commit a14c4
-
Ping @florent
-
Same issue here, i posted on discord about it but perhaps the forum is a better place for it.
I see the the newer releases of XO from source adds CBT to backups and enables it by default. Anyone else having issues with this feature? All my VMs were stuck "exporting content" in my test environment this morning, also looping tasks consisting of "call: sr.getAllUnhealthyVdiChainsLength" and under the health dashboard all my VDIs were unhealthy needing to coalesce.
What fixed it for me was to roll back to a pre CBT release, and remove all the .cbtlog files from my SRs, i had to delete all my VM snapshots, let garbage collection run, then ran a fresh backup run.
-
-
Don't know how relevant it is. But I haven't seen any of the tasks exceed 24h. So either they're timing out and being removed silently after a day has passed.
Or something else is happening. -
That's a normal XAPI behavior: tasks are destroyed after 24h by default in XCP-ng
-
Hi,
We added 2 fixes in latest commit on master, please restart your XAPI (to clear all tasks), and try again with your updated XO
-
I am seeing very good results right now after updating.
Looks good to me at least. I'll also be able to verify at my other lab to see if things are fixed there too, tomorrow.Hoping that OP also sees similar good results as well.
-
I will get my instance updated and give it a go then report back
-
This issue seems to be fixed. Tasks are finishing and disappearing. I am seeing the below errors but I believe that is being discussed in another thread possibly somewhere.
-
@Delgado said in Stuck VDI Exports After Latest Upgrade:
This issue seems to be fixed. Tasks are finishing and disappearing. I am seeing the below errors but I believe that is being discussed in another thread possibly somewhere.
Hi,
can you upload the full log of this error ? I am still not reproducing it in our lab
regards
-
@florent I'll see if I can find the log from this job and attach it to this thread. Of note I am now on commit 253aac4eaa806c552ae492d42a09eb21e7124943 and currently not seeing the error but I had just turned NBD back on. Never the less I will look for the log. Thanks!
-
@florent Here is the log from that job. 2024-06-27T18_11_00.579Z - backup NG.json.txt
