F
Offline
On tomorrow 's release we will:
following this we will add core xo metrics to the openmetrics ( prometheus) export to allow users and support to dig through real world data when needed
Cancelling a backup run is also on our roadmap ,as soon as we have done reworking the tasks to use xo-task on the full chain, and not an hybrid of various iteration of the tasks objects
better tracking of the backup run is also on our roadmap , this is also linked to the task changes plus are some issues with the size on the remote before being able to do this
In any case, both are on our backlog
@Pilow said in Veeam backup with XCP NG:
@olivierlambert I get you.
Same in veeam and synthetic fulls, it's only between the server/proxy and the repository.
but we get a progressing percentage while the job sits while doing the synthetic full.would be cool to "see" the progress of merging.
why ? bad habits from previous technologies used I guess.
Giving better observability is something we are working actively on , this is tightly coupled with being able to stop a task, and I hope we will be able to show the progress soon
@manilx yes, that's it
we should change the "full backup interval" to "base/complete backup interval", to clarify what is a full . And we will do it.
we are reworking the import code , along the new backup code ( https://xcp-ng.org/forum/topic/10664/our-future-backup-code-test-it ) to unify the differetn disk transformation path
For now:
@andrewreid Yes, I can't wait to share it with everybody.
@AlexD2006 thanks for signaling this, we just merged a fix, that fix it on our labs
can you test it on your side ?
https://github.com/vatesfr/xen-orchestra/pull/9202
@AlexD2006 this looks like the right issue, we will look into it immediately ( yesterday was a public holiday in France)
@Andrew then again, with such a precise report, the fix is easier
the fix should join master soon
found it , there was not one, but 2 bugs
albeit small they were putting the xo backup in a incorrect state
the fix is here, and will be merged as soon as possible and then released as a patch for xoa users
https://github.com/vatesfr/xen-orchestra/pull/8780
really thank you everyone for the information
@pedro_udifar object lock works quite well with xo
it is automatically detected if the s3 user has enough privilege to see if object lock is enabled
the main point is that it won't create "cache.json.gz" files since theses files must be updated, so the backup listing is a little slower
note that XO retention should be at least 1 more than object lock . At worst you will have error , saying that XO coudn't delete an old backup
The fix is on a separated branch : fix_s3_nonencrypted_to_encrypted_xva ( PR : https://github.com/vatesfr/xen-orchestra/pull/10061 )
@pedro_udifar we have found a clue , if the destination is encrypted and the source is not
can you test this branch ? https://github.com/vatesfr/xen-orchestra/pull/10061
if you are using a xoa, we can deploy the fix if you open a support tunnel
now that I am in front of the source code , it is already getting the real size, so it should compute the right size
are the source or destination encrypted ?
Thanks for the detailed write-up, Pedro.
I'm not a backup expert, far from it. but you might be right that https://github.com/vatesfr/xen-orchestra/pull/9396 only fixed the size estimation for direct full backups, not the mirror path.
Your error is the same as the maximum size allowed one, just twelve bytes over 209715200000. Before this turns into a GitHub issue, it would help to know whether it reproduces on a fresh mirror job and whether bumping minPartSize actually cleared it, so we can be sure it's the mirror code and not the B2 remote settings.
The object storage docs list Backblaze B2 as supported (https://docs.xen-orchestra.com/xo5/object-storage-support#supported-storage-providers) but don't say much about part-size tuning.
If it's awkward to test in isolation, a mention to @Team-XO-Backend is probably the quickest route, since they own the backup job code.
The XVA checksum warnings in your second screenshot look like a separate non-blocking clean VM directory step rather than the cause.
I hope that points somewhere useful!
nice catch
and the mirror path is easier to fix . Thaks @pedro_udifar and @poddingue we will fix it asap
will it be possible to test a potential fix @pedro_udifar , since it's often specific to a provider ( and need huge VM to work )?
the new code is now in master
@Andrew working on it with @julienxovates
at least including them by default , without following them, in the archive seems doable . The change on the XO5 UI may be trickier, and we are not far to rewrite it for XO6
@Andrew good news : the lvm id/name collision is the root cause of a lot of issues of the the file restore.
I think restoring the symbolic links will be hard to secure ( a symbolic link can point anywhere in the os and the disk are mounted in the xoa as root) and quite tricky to ensure we are not bombing ourself
I think we can add them to the zip/tar as (brokenà symbolic link, but we need to look into the impact on the client side file browser
@Andrew thank
I pushed some fixes , can you retry the VM . With this config in the toml ? if it fails if will gives us more logs in journalctl
[logs]
# Display all logs matching this filter, regardless of their level
filter = 'xo:backups:RemoteAdapter'
@ravenet not directly, it's not mountable into linux
But if ( and this is a big if) we are able to make FLR more resilent, I have some next step that would expose the disks so you can mount it directly in your preferred OS or XCP , probably through webdav
we did a lot of ground work on the file restore front , and have plan to continue working on it.
What is testable and should work :
Note that, by design, the file restore is slow : each time the xoa want to read 4KB of data or even 1B we have to load a full backup block (2MB) decrypt, and inflate it ,extract the 4KB , ... . Expect 1/10 to 1/30 of your remote performance
But at least it should not crash anymore, with zip or tar restore. The worse we tested is a windows folder ( 26GB , 10 of thousands of files)
For now the fixes are in https://github.com/vatesfr/xen-orchestra/pull/9776 ( branch fix_flr ) and are expected to reach master during next week