@andrewreid Yes, I can't wait to share it with everybody.

@b-dietrich said in Externalised backup LTO:

Hi everyone,

I would like to know if it's possible to externalised backup on library tape with XOA ?

Is it in the roadmap for 2024 ?

I will let @olivierlambert on the backlog point. It is still planned, but there is a lot of ground work before :

• since tape can't be easily seek , so you'll have to write the backup at once no turning back to update a previously written block/metadata
• you'll have to build a catalog of tape<->backup to know which tape you'll have to use to restore, or rewrite. This is a huge change since XO don't use any database , the backup repositories are self contained : you can mount a backup repository to a new XO, and the backup will be listed . With tapes, you'll have to keep (and backup) the backup catalog. Yep , we'll need to backup the backups and ensure it's recoverable

That being said, the mirror backup feature as been built to pave the way to tape backup

For now the easiest way to do tape backup is to use full backup to a backup repository only used for this, and to mirror it to tapes. At our scale, priorities can also change if there is a big enough sponsor, that is ready to take a part of the financial load of this feature and gives us access to real world hardware and processes.

@CJ yes we have to fix the preview panel

the test with @dumarjo showed that there is still a bug during the import. I am still investigating it and will keep you informed, hopefully today or tomorrow

@ismo-conguairta said in VMware migration tool: we need your feedback!:

I have two different behaviour on two different XO instances. Each XO instance refers to a different pool (different hosts, same xcp-ng version). In both the instances I try to connect to the same Private Virtual Datacenter based on VMware/vSphere at OVH.

In the first one I get the following error message by using the web UI: "invalid parameters" (take a look at this logfile 2023-02-28T19_25_21.933Z - XO.txt )

In the second one, I get the following error message by using the web UI "404 Not Found https://<vsphere-ip>/folder/<vm-name>/<vm-name>.vmx?dsName=<datastore-name>"

By using the xo-cli I get the "404 Not Found" on both the instances.

Regarding the "404 Not Found", I want to point out that at OVH I have a VMware datacenter (with 2 hosts) and in order to access to the storage I need to specify the parameter dcPath=<datacenter-name>

So the right URL should be https://<vsphere-ip>/folder/<vm-name>/<vm-name>.vmx?dcPath=<datacenter-name>&dsName=<datastore-name>

Simply adding (in a static way) the dcPath specification on line :54 of esxi.mjs file makes it work.

I thought it was constant. I will look into the api to get it, and if not possible expose it in the UI

@Seclusion : noted I will look into theis error message, this one is a first for me

@brezlord mac address and uefi should works now

@rochemike patch done this morning

@rochemike great, that will be even easier

can you open a support ticket and open a support tunnel ? I will connect and patch your installation

@acomav you're up to date on your XOA

I pushed a new commit , fixing an async condition on the fix_xva_import_thin branch . Feel free to test on your XO from source.

@techjeff thanks for your effort, I found the problem

can you test this branch : fix_cr_healthcheck ? (git checkout fix_cr_healthcheck from xen-orchestra folder) it will be merged soon

@afk the agent is as dumb as possible

also if you encrypt the backup, the agent will need to decrypt the metadata to detect the chains, thus having access to the encryption key, which need getting the encryption key out of XO and transferred to the immutability agent

I think it will be easier to provide more feedback on the immutabiltiy backup, XO has access to the chain , and / or alert when something seems to be strange

@peo do you have anything in the xo logs ? can you post the json of a failed backup job ?

@peo in object storage world ( S3 compatible) , there is not really the notion of "directory", it's more a convention that / is a logical level in the file key, but rclone has errors on "Directory not empty". It let me think that rclone do things a little differently than aws s3.

We try to be as compatible as possible with S3 implementation, but each object storage has its own quirk. Thus we only supports the most used setups for now.

@SudoOracle could you post the full json log of the backup ?
(you can get it by clicking on the download button on top of a failed execution job)

if possible, one per type of failed backup job

@manilx are there multiple job running on the same VM ?

this happens when 2 backup jobs are working on the same VM , or a backup and a background merge worker .
If it's the first case, you can use a job sequence to ensure the backup run one after another

If it's the latter, you have a check box to ensure the merge is done directly inside the job (bottom of the advanced settings of the backup job ), instead of doing this in a background job. It will slow down a little the job, but ensure it's completely done.

@abudef
incremental backups on block storage are compressed with brotli in fastest mode. But this is enough to have a massive gain , since we compress all the zeroes.

incremental backup ( *.vhd files ) on non block storage aren't compressed

Full backup (as in xva file ) aren't compressed by xo, but can be compressed by xcp ng ( using gzip or zstd )

for now, this is not configurable, we are waiting for the future remote form to handle this.

@afk the newer VMFS put more lock on the files, locking the full chain of snapshot and base disks instead of locking only the active disk.
Even VMFS5 sometimes lock the full chain.
The more open storage is a NFS, that can additionally be accessed directly by XO, not going through the soap api , giving anice performance boost and not hammering the esxi too much ( only for the metadata )

We don't intend to migrate the VM running in vmware from XO automatically. A user could script it though, by using a combination of the vmware apis and xo-cli.

We never succeed in getting the disks though NBD, but it should be possible ( https://github.com/vexxhost/migratekit/blob/main/internal/vmware_nbdkit/vmware_nbdkit.go#L91 ) and ( https://vdc-download.vmware.com/vmwb-repository/dcr-public/8ed923df-bad4-49b3-b677-45bca5326e85/d2d90bb6-d1b3-4266-8ce5-443680187a9a/vim.vm.device.VirtualDevice.BackingInfo.html )
What we are mostly missing here, is internal knowledge on the Vmware side : how to get the NBD server address,how to authenticate and how to get the exportname of a disk through the soap API .

Since we already have the knowledge of using NBD to read massive volume of data as long as we can connect.

they are created to ensure the samba mount is not removed by the os or another thread on XO
I would not delete them , especially since they are 0bytes files

@rtjdamen said in Feedback on immutability:

@florent so this does mean it will never work when a forever incremental is used?

you can't have a immutable forever backup without having a infinite length, and an infinite

It may be possible only if we release the constraints.
The immutable script could release the immutability , merge the disks, but that means : the immutability will be lifted from time to time, and the responsibilities of the immutability script will be greater, and we'll need a way to track the vhd to merge and transmit the information to the immutability script