@Pilow said in Veeam backup with XCP NG:

@olivierlambert I get you.

Same in veeam and synthetic fulls, it's only between the server/proxy and the repository.
but we get a progressing percentage while the job sits while doing the synthetic full.

would be cool to "see" the progress of merging.
why ? bad habits from previous technologies used I guess.

Giving better observability is something we are working actively on , this is tightly coupled with being able to stop a task, and I hope we will be able to show the progress soon

@manilx yes, that's it

we should change the "full backup interval" to "base/complete backup interval", to clarify what is a full . And we will do it.

we are reworking the import code , along the new backup code ( https://xcp-ng.org/forum/topic/10664/our-future-backup-code-test-it ) to unify the differetn disk transformation path

For now:

• V2V handles flat, cowd, sesparse, stream optimized vmdk in the format "metadata.vmdk and a data file ) and ova ( for VSAN)
• disk import support monolithic vmdk ( cowd, sesparse, streamoptimized )
• ova improt support only stream ova with streamoptimzed vmdk inside

@andrewreid Yes, I can't wait to share it with everybody.

@Andrew then again, with such a precise report, the fix is easier

the fix should join master soon

found it , there was not one, but 2 bugs
albeit small they were putting the xo backup in a incorrect state
the fix is here, and will be merged as soon as possible and then released as a patch for xoa users
https://github.com/vatesfr/xen-orchestra/pull/8780

really thank you everyone for the information

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(xapi): respect the config file setting for preferNbd #8780

@RobWhalley saved the day
We are still working on it to better understand why it fixes and improve the situation

@RobWhalley nice work, I am testing it right now

@olivierlambert Nice catch
CBT will only be enabled when using purge data , I will fix the first one

(this is because CBT does not improve things without purge snaphot data , but has some edge case , so at least we won't enable it for nothing )

@olivierlambert @Gheppy that is a nice catch and it gives us a interesting clue

I am currently working on it

@MajorP93 this settings exists (not in the ui )

you can create a configuration file named /etc/xo-server/config.diskConcurrency.toml if you use a xoa

containing

[backups]
diskPerVmConcurrency = 2

@MajorP93
interesting, Note that this is orthogonal to NBD.
I note that there is probably more work to do to improve the performance and will retest VM with a lot of disk
Performance is really depending on the underlying storage.
compression and encryption can't be done in "legacy mode" , since we won't be able to merge block in place in this case.

@acebmxer is 10.120.20.119 your xo- proxy ?

I misread your question, you want to exclude the xoa from the process, not the proxy. Today the disk conversion can only be done directly by the xoa.
The data flow is always {vpshere/esxi} => xoa => xcp-ng , with a proxy that can be between xoa and xcp-ng

The only way to handle this without goiing through delilah would be to deploy a second xoa/xoce directly in tilton pool . You can have multiple xoa connected to the same pool.

interesting

can you try to do a perfomance test while using block storage ?

This will store backup are multiple small ( typically 1MB ) files , that are easy to deduplicated, and the merge process will be moving / deleting files instead of modifying one big monolithic file per disk. It could sidestep the hydratation process.
This is the mode by default on S3 / azure, and will probably be the mode by default everywhere in the future, given its advantages

(Note for later : don't use XO encryption at rest if you need dedup, since even the same block encrypted twice will give different results)

@olivierlambert the tar stream is passed as is
is there any warning in the relevant backup job log ?

@acebmxer is the proxy used as a http proxy forone of the involved pool ?
can you show a screenshot of your "server" screen ?

@Austin.Payne a health check will download the full backup from your remote to the storage repository
what is the size of the VM ?

@planedrop you can't change the remote encryption if the remote is not empty

in the future we intend to be able to use rolling encryption ( that is encrypting the new block/file with the new key ) to permit an easier upgrade and key rotation

@MBNext is it possible to post the full json (you can download it on the top of the windows that shows the backup progress )

@MajorP93 the size are different between the disks, did you modify it since the snapshots ?

would it be possible to take one new snapshot with the same disk structure ?