@andrewreid Yes, I can't wait to share it with everybody.
florent
@florent
Best posts made by florent
-
RE: Backblaze B2 as a backup remote
-
RE: Externalised backup LTO
@b-dietrich said in Externalised backup LTO:
Hi everyone,
I would like to know if it's possible to externalised backup on library tape with XOA ?
Is it in the roadmap for 2024 ?
I will let @olivierlambert on the backlog point. It is still planned, but there is a lot of ground work before :
- since tape can't be easily seek , so you'll have to write the backup at once no turning back to update a previously written block/metadata
- you'll have to build a catalog of tape<->backup to know which tape you'll have to use to restore, or rewrite. This is a huge change since XO don't use any database , the backup repositories are self contained : you can mount a backup repository to a new XO, and the backup will be listed . With tapes, you'll have to keep (and backup) the backup catalog. Yep , we'll need to backup the backups and ensure it's recoverable
That being said, the mirror backup feature as been built to pave the way to tape backup
For now the easiest way to do tape backup is to use full backup to a backup repository only used for this, and to mirror it to tapes. At our scale, priorities can also change if there is a big enough sponsor, that is ready to take a part of the financial load of this feature and gives us access to real world hardware and processes.
-
RE: VMware migration tool: we need your feedback!
the test with @dumarjo showed that there is still a bug during the import. I am still investigating it and will keep you informed, hopefully today or tomorrow
-
RE: VMware migration tool: we need your feedback!
@ismo-conguairta said in VMware migration tool: we need your feedback!:
I have two different behaviour on two different XO instances. Each XO instance refers to a different pool (different hosts, same xcp-ng version). In both the instances I try to connect to the same Private Virtual Datacenter based on VMware/vSphere at OVH.
In the first one I get the following error message by using the web UI: "invalid parameters" (take a look at this logfile 2023-02-28T19_25_21.933Z - XO.txt )
In the second one, I get the following error message by using the web UI "404 Not Found https://<vsphere-ip>/folder/<vm-name>/<vm-name>.vmx?dsName=<datastore-name>"
By using the xo-cli I get the "404 Not Found" on both the instances.
Regarding the "404 Not Found", I want to point out that at OVH I have a VMware datacenter (with 2 hosts) and in order to access to the storage I need to specify the parameter
dcPath=<datacenter-name>
So the right URL should be https://<vsphere-ip>/folder/<vm-name>/<vm-name>.vmx?dcPath=<datacenter-name>&dsName=<datastore-name>
Simply adding (in a static way) the dcPath specification on line :54 of esxi.mjs file makes it work.
I thought it was constant. I will look into the api to get it, and if not possible expose it in the UI
@Seclusion : noted I will look into theis error message, this one is a first for me
-
RE: VMware migration tool: we need your feedback!
@brezlord mac address and uefi should works now
-
RE: Xen-Orchestra Terraform provider and Windows
@rochemike patch done this morning
-
RE: Xen-Orchestra Terraform provider and Windows
@rochemike great, that will be even easier
can you open a support ticket and open a support tunnel ? I will connect and patch your installation
-
RE: Import from VMware fails after upgrade to XOA 5.91
@acomav you're up to date on your XOA
I pushed a new commit , fixing an async condition on the fix_xva_import_thin branch . Feel free to test on your XO from source.
-
RE: Continuous Replication job fails "TypeError: Cannot read properties of undefined (reading 'uuid')" at #isAlreadyOnHealthCheckSr
@techjeff thanks for your effort, I found the problem
can you test this branch : fix_cr_healthcheck ? (
git checkout fix_cr_healthcheck
from xen-orchestra folder) it will be merged soon
Latest posts made by florent
-
RE: Problems doing live/warm migrations from vSphere/ESXi 7 to XCP-ng
@rfx77 said in Problems doing live/warm migrations from vSphere/ESXi 7 to XCP-ng:
Thanks. The problem is taht we dont have a NFS datastore to use. So we had to do export/import.
you can migrate without a NFS datastore, but on esxi 6.5+ , you'll need to shutdown the VM before starting the migration
and with vsan there is an additionnal , slower steps : extractig the vmdk to a Remote (in XO ) -
RE: Import from VMWare fails Error: 404 Not Found
@sashasaturn said in Import from VMWare fails Error: 404 Not Found:
Type: vCenter Server with an embedded Platform Services Controller
Product: VMware vCenter Server Appliance
Version: 6.7.0.55000
Build number: 22509723
XOA Current version: 5.92.1 - XOA build: 20240401
Vcenter server with two data centers (DC1 and DC2), each has two clusters (DC1CL1, DC1CL2, DC2CL1, DC2CL2)TESTVM1 located in DC2CL1
I'm using VCenter administrator account with full access to everything to import VM from VMWare to XOA
VM import from VMWare fails with an error:
vm.importMultipleFromEsxi { "concurrency": 1, "host": "VCENTER_IP", "network": "b46e5d3e-1c9a-0ad0-c2fc-fed488fd870b", "password": "* obfuscated *", "sr": "db128e81-9efe-76d1-5185-71fa15c79a2f", "sslVerify": false, "stopOnError": true, "stopSource": false, "user": "VCENTER_ADMIN", "vms": [ "vm-81111" ] } { "cause": { "size": 0 }, "succeeded": {}, "message": "404 Not Found https://VCENTER_IP/folder/TESTVM1/TESTVM1.vmx?dcPath=DC1&dsName=storage_name", "name": "Error", "stack": "Error: 404 Not Found https://VCENTER_IP/folder/TESTVM1/TESTVM1.vmx?dcPath=DC1&dsName=storage_name at Esxi.#download (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/vmware-explorer/esxi.mjs:95:21) at Esxi.download (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/vmware-explorer/esxi.mjs:113:21) at Esxi.getTransferableVmMetadata (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/vmware-explorer/esxi.mjs:273:17) at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:158:22) at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:141:20) at MigrateVm.migrationfromEsxi (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/migrate-vm.mjs:178:28) at file:///usr/local/lib/node_modules/xo-server/src/api/vm.mjs:1432:30 at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:158:22) at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:141:20) at asyncEach.concurrency.concurrency (file:///usr/local/lib/node_modules/xo-server/src/api/vm.mjs:1430:11)" }
I'm guessing, but the URL https://VCENTER_IP/folder/TESTVM1/TESTVM1.vmx?dcPath=DC1&dsName=storage_name" looks incorrect because I think dcPath=DC2
Thanks,
Alex
hi, there is a fix to better handle mulitple datacenter merged in master this morning.
Can you update a retest ? -
RE: Why does the backup use snapshots and not CBT
@rtjdamen said in Why does the backup use snapshots and not CBT:
@olivierlambert i am currently looking at Quadric Software and storware, Quadric Software is doing the cbt backup exactly as needed, i got there description on the workflow yesterday
*When you run an ABD/hypervisor backup, the A3 will snapshot the VM for the duration of the backup data acquisition phase. After the datea is collected, the snapshot is immediately deleted.
In the case of CBT backups, the snapshot is deleted with the special XAPI command vdi-data-destroy, which frees data associated with the snapshot, but retains metadata, including the block allocation for the disk. This allows us to perform a delta backup the next time, with extremely negligible storage overhead.*
This is what we need
we did a poc of it
we still need a snapshot ( or to stop the VM), bu at least we won't keep it
the hard part is that we need to change the way we store metadata : today it is stored at the VM snapshot level, but this snapshot is deleted after a CBT enabled backup
Hopefully it will be done this year -
RE: Updated XOA with kernel >5.3 to support nconnect nfs option
@Forza said in Updated XOA with kernel >5.3 to support nconnect nfs option:
@florent hi, for the full backup test above, I used normal mode with zstd enabled. There were no snapshots of the source VM and it was stored on local ssd storage on the pool master.
can you test a backup on an empty remote with block mode ?
-
RE: Cannot create property 'ctkEnabled' on string 'true' on trying to import from ESXi
@parsedave said in Cannot create property 'ctkEnabled' on string 'true' on trying to import from ESXi:
@Danp do you know when the fix will be integrated in to an update?
Thanks
DaveIt is merged now, it will be released in XOA latest tomorrow
-
RE: Updated XOA with kernel >5.3 to support nconnect nfs option
On the read side :
- the legacy mode : xapi build an expor from the vhd chain
- NBD : we read individual block on the storage repository
On the write Side :
- the default file mode : we write 1 big file per disk
- block mode : we write a compressed block per 2MB data
We don't have a lot of room on the legacy mode. The NBD + block gives us more freedom, and , instinctively , should gain more from nconnect, since we will read and write multiple small blocks in parallel
What mode are you using ?
@Forza said in Updated XOA with kernel >5.3 to support nconnect nfs option:
I did a 3 x backup of a single VM hosted on an SSD drive on the same host as XOA is running, which is also pool master.
This is with
nconnect=1
:
This is with
nconnect=16
:
The transfer speed according to XOA is slightly less, but looking at the bandwidth graph, it looks like the LACP bonded network on the storage server is reaching a higher max throughput.
I will test some more with incremental backups and see if there's a difference with them.
If we ignore the
nconnect
for a second, and just look at the graphs, it seems we have a lot of possibilities to improve the backup performance if we could make the transfer more even. What is causing this type of pattern? I do not believe any coalesce was happening during this test. -
RE: Cannot create property 'ctkEnabled' on string 'true' on trying to import from ESXi
@parsedave Hi,
We worked on a branch that should fix it : https://github.com/vatesfr/xen-orchestra/pull/7583 It should be merged in master by the end of the week Thanks for your report
-
RE: Two questions in regard to XO and remotes
@HolgiB said in Two questions in regard to XO and remotes:
Hey there,
I have two questions in regard to remotes:
- How does XO handle backup jobs to multiple remotes ? In example if I have attached and configured two nodes for a backup job. Are the files stores parallel to both remote locations ? Or are they stored sequentially to remote "A" and then to remote "B" ?
- What it the best practice (so to speak) for migrating existing backups to a new location ?
Simply sync them over to the new remote, configure the new remote to each backup job and remove the old remote ?
Thanks in advance !
- Holger
1- they are stored in parallel, so it goes as slow as the slower remote.
2. For a migration : copy/rsync does the job, that way it won't change the uuid. If you want to have a daily replication of a remote, you can use a mirrror backup. -
RE: Health Checks Failing
@Delgado said in Health Checks Failing:
@florent can confirm that health checks are working again! Thanks for the prompt reply!
thanks for the precise bug report, that is half of the job done
-
RE: Health Checks Failing
@Delgado said in Health Checks Failing:
_MixinXapiWriter
Hi,
This is probably linked to the PR that makes healthcheck timeout configurable, I am looking into it right nowRegards