On tomorrow 's release we will:

• reduce the polling for the patches list
• add more debug to know exactly what request go through the xo6 api (rest api)

following this we will add core xo metrics to the openmetrics ( prometheus) export to allow users and support to dig through real world data when needed

@Forza

Cancelling a backup run is also on our roadmap ,as soon as we have done reworking the tasks to use xo-task on the full chain, and not an hybrid of various iteration of the tasks objects

better tracking of the backup run is also on our roadmap , this is also linked to the task changes plus are some issues with the size on the remote before being able to do this

In any case, both are on our backlog

@Pilow said in Veeam backup with XCP NG:

@olivierlambert I get you.

Same in veeam and synthetic fulls, it's only between the server/proxy and the repository.
but we get a progressing percentage while the job sits while doing the synthetic full.

would be cool to "see" the progress of merging.
why ? bad habits from previous technologies used I guess.

Giving better observability is something we are working actively on , this is tightly coupled with being able to stop a task, and I hope we will be able to show the progress soon

@manilx yes, that's it

we should change the "full backup interval" to "base/complete backup interval", to clarify what is a full . And we will do it.

we are reworking the import code , along the new backup code ( https://xcp-ng.org/forum/topic/10664/our-future-backup-code-test-it ) to unify the differetn disk transformation path

For now:

• V2V handles flat, cowd, sesparse, stream optimized vmdk in the format "metadata.vmdk and a data file ) and ova ( for VSAN)
• disk import support monolithic vmdk ( cowd, sesparse, streamoptimized )
• ova improt support only stream ova with streamoptimzed vmdk inside

@andrewreid Yes, I can't wait to share it with everybody.

@AlexD2006 thanks for signaling this, we just merged a fix, that fix it on our labs
can you test it on your side ?
https://github.com/vatesfr/xen-orchestra/pull/9202

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(backups): getOldEntries must accept entries without id or timestamp #9202

@AlexD2006 this looks like the right issue, we will look into it immediately ( yesterday was a public holiday in France)

@Andrew then again, with such a precise report, the fix is easier

the fix should join master soon

found it , there was not one, but 2 bugs
albeit small they were putting the xo backup in a incorrect state
the fix is here, and will be merged as soon as possible and then released as a patch for xoa users
https://github.com/vatesfr/xen-orchestra/pull/8780

really thank you everyone for the information

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(xapi): respect the config file setting for preferNbd #8780

@ItzJay in your journalctl -i xo-server log you should have something like nbdkit logs of ${diskPath} are in ${tmpDir} where tmpDir is something like /tmp/xo-serverXXXX/stderr

can you post the file ?

alternatively, you can open a support ticket

thanks Andryi
We us round robin when using NBD , but to be fair, it does not change the performance a lot in most of the case. The concurrency settings ( multiple connection to the same file ) is helping when there is a high latency between XO and the host.

SO , @fcgo if you have thousand of VMs , you should enable NBD it will consume less resource on the DOM0 and XO , and it will be spread on all the possible hosts.

@deligatedgeek are you modifying the chunk for delta or full backup ?
because a lot of part in the incremental code assume that blocks are aligned to 2MB

For full backup ( one big xva file per VM) the code should handle file with a size bigger than 4GB , by dynamically increasing the chunk size. We do not increase the chunk size by default because it also increase a lot the memory consumption. The minimum size of a chunk is given by MIN_PART_SIZE ( 5MB as per AWS doc ) and the maximum size is MAX_PART_SIZE ( 5GB) . The maximum file is 50TB , the biggest full backup I saw in the wild was a few hundred GB

@ItzJay said in Problems import vm Vmware to Xcp-ng from import Vmware and OVA:

@florent By base OS you mean the based Debian XOA appliance?

yes , I was not sure it was a XOA. If it's a xoa, how old is it ? would it be ok to open a ticket so we can check it on monday ?

@olivluca what are your base os ? because on a XOA, I thik gnutls ins installed, so nbdkit is compiled with tls supports

@flakpyro the double close will be improved but this is only a log issue (we close the file twice to ensure it is closed in the nominal case and in case of error )
we did the minimal patch to ensure the root issue is addressed before weekend

patch is released

@flakpyro we did found a race condition in our code

We will do a complete post mortem after , but for now :

• the fix is in master ( https://github.com/vatesfr/xen-orchestra/commit/53f5c63fa242e398caab583223719741cd25bb1d )
• we are preparing a patch that will be release as soon as we are done with QA

0 spacotte-vates committed to vatesfr/xen-orchestra

fix(backups): race condition on merge (#9542)

@Pilow we really did only add one small log . we'll need to dive deeper for a real cause ( and fix)

Let me create a card do check this hypothesis ( and do a the fix @andrew )