the new code is now in master

@Andrew working on it with @julienxovates

at least including them by default , without following them, in the archive seems doable . The change on the XO5 UI may be trickier, and we are not far to rewrite it for XO6

@Andrew good news : the lvm id/name collision is the root cause of a lot of issues of the the file restore.

I think restoring the symbolic links will be hard to secure ( a symbolic link can point anywhere in the os and the disk are mounted in the xoa as root) and quite tricky to ensure we are not bombing ourself

I think we can add them to the zip/tar as (brokenà symbolic link, but we need to look into the impact on the client side file browser

@Andrew thank

I pushed some fixes , can you retry the VM . With this config in the toml ? if it fails if will gives us more logs in journalctl

[logs]
# Display all logs matching this filter, regardless of their level
filter = 'xo:backups:RemoteAdapter'

@ravenet not directly, it's not mountable into linux

But if ( and this is a big if) we are able to make FLR more resilent, I have some next step that would expose the disks so you can mount it directly in your preferred OS or XCP , probably through webdav

we did a lot of ground work on the file restore front , and have plan to continue working on it.

What is testable and should work :

• restore from partition of types :
  • ext2 / ext3 / ext4
  • XFS
  • Btrfs
  • FAT / VFAT / exFAT (in-kernel since 5.7)
  • NTFS3 (new in-kernel driver, since 5.15 — parallel to ntfs-3g)
  • f2fs
  • ISO 9660 / UDF (optical media)
  • squashfs / erofs (read-only images)
• restore from LVM as long as there is only one VDI involved
• handle disk without partition table, with GTP, with MBR

Note that, by design, the file restore is slow : each time the xoa want to read 4KB of data or even 1B we have to load a full backup block (2MB) decrypt, and inflate it ,extract the 4KB , ... . Expect 1/10 to 1/30 of your remote performance

But at least it should not crash anymore, with zip or tar restore. The worse we tested is a windows folder ( 26GB , 10 of thousands of files)

For now the fixes are in https://github.com/vatesfr/xen-orchestra/pull/9776 ( branch fix_flr ) and are expected to reach master during next week

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(backups): file level restore : omnibus fixes #9776

@tsukraw multiple NBD connection will open multiple reading connection, but the writing one is always one stream per disk in incremental replication

with full replication, it's one stream for read and one for write

@acebmxer back to work
thank you for yor patience and help on this. I feel that it's not the same issue , with abrupt increase

W will try our best to also fix this one

@laszlobortel since we are generating the stream sequentially, we though that there wasn't any real world gain. Also, if my understanding is correct using one thread means we will have only one process( + xo + nbdkit-vddk) processes , and we were trying to limit the risk of zombie process locking the disks , which was far too common with the previous code

we'll do a quick test to ensure it does not beak anything and propose either a better default of a configuration option

note that this will be a concurrency per disk, so VM with a lot of disk will put more pressure on the source

@julienXOvates PR is here https://github.com/vatesfr/xen-orchestra/pull/9868

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(xo-web): ignore the number of snapshot of replicated VM in the health dashboard #9868

also do you have a reverse proxy/ http proxy in front of xo ? it can block the bigger upload by default

@nasheayahu qcow2 disk import has been merged recently ( https://github.com/vatesfr/xen-orchestra/pull/9817 ) , but there is a caveat

the clusters in the disk must be in order to work . This is generally the case for a disk exported by any system, but not the case for a disk used in production that you want to import

where does this disk comes from ? if you have access to your SR from the outside, you can also put the qcow2 file directly , by renaming it as a valid uuid ( I think uuidgen can do this ) and putting it directly in the SR/uuid repository

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed feat(xo-web/xo-server): import qcow2 disk from form #9817

@poddingue this is something that was hidden with the previous system ( same disk chains, but not shown as snapshot )

@julienxovates are you ok to not check the vm tagged as replication from this chech ?

the last rewrite of the stream processing ( spring 2025 ) focused on stability and memory footprint, and , on a standard cpu, it tops at around 300MB/s per backup job. Your benchmarks are very interesting, and they confirm most of it.

this limit was not really an issue since, in most case the xapi was limiting around 100MB/s per disk , but it will be more a more visible limit

Note that master have some fixes on the memory usage (not related to backups)

That's why we have started an internal workforce focused on performance, with all the teams from the kernel to the backups, including storage, network and xapi.

If I can brag a little :

i9 , nvme disk , backup to a nvme disk in passthrough, xoa and vm are on the same host, so it's quite far from real world data, but it shows where the limit is

@tsukraw I am taking the ticket and will keep you informed as soon as possible

@ducatijosh did you do a yarn build ?

@bvivi57 xo do check license if you have xostor installed, since it need some magic to work at a lot of steps ( like the rolling pool updates)

this is the expected behavior with a manually installed xostor ( cc @julienxovates for information )

@acebmxer not yet
(a little under the water with the release patch, but we will do it )

@DustyArmstrong for now the rest api doesn't support server side sorting ( we don't have the memory budget to do this )

it will be improved in the future

@bvivi57 said:

getLicenses

do you have xostor installed ?