Subcategories

  • VMs, hosts, pools, networks and all other usual management tasks.

    355 Topics
    3k Posts
    H
    Hi dinhngtu I think i have found the problem. The end of the ssh key has 'bastion production' at the end of the key and i believe this is the wrong format for cloud init, all of the other parts of the configuration file work as xen orchestra indicated that the guest utils are installed. It a bit of a pain to regenerate the keys because i have a lot of devices i connect too.
  • ACLs, Self-service, Cloud-init, Load balancing...

    88 Topics
    756 Posts
    A
    @olivierlambert I’ve got a few VM’s that lock up from time to time. There is no rhyme nor reason (different hosts, different storage, etc). As a result, I just have a job that restarts them at 5:30 am every day. I’d like to have the software tool software restart them but, in the occasion wher the VM is locked up tight as Fort Knox, I’d like the hard reset to happen. Make sense ?
  • All XO backup features: full and incremental, replication, mirrors...

    354 Topics
    3k Posts
    A
    Last night 2 of my vms failed to complete a delta backup. As the tasks could not be cancled in any way i rebooted XO (built from sources) the task still show "runing" so i restarted the tool stack on host 1 and the tasks cleared. I attempted to restart failed backups and again the backup just hangs. It create the snapshot but never transfer data. The Remote is the same location as the nfs storage the vms are running from. So i know the Storage is good. A few more rebooted of XO and tool stack. I rebooted both host and each time backups get stuck. If i try to start a new backup (same job) all vms hang. I tried to run a full delta backup and same. I tried to update XO but I am on current master build as of today (6b263) I tried to do a force update and still backup never completes. I built a new VM for XO and installed from sources and still fail. [image: 1750513903129-screenshot-2025-06-21-094543.png] Here is one of the logs from the backups... { "data": { "mode": "delta", "reportWhen": "always", "hideSuccessfulItems": true }, "id": "1750503695411", "jobId": "95ac8089-69f3-404e-b902-21d0e878eec2", "jobName": "Backup Job 1", "message": "backup", "scheduleId": "76989b41-8bcf-4438-833a-84ae80125367", "start": 1750503695411, "status": "failure", "infos": [ { "data": { "vms": [ "b25a5709-f1f8-e942-f0cc-f443eb9b9cf3", "3446772a-4110-7a2c-db35-286c73af4ab4", "bce2b7f4-d602-5cdf-b275-da9554be61d3", "e0a3093a-52fd-f8dc-1c39-075eeb9d0314", "afbef202-af84-7e64-100a-e8a4c40d5130" ] }, "message": "vms" } ], "tasks": [ { "data": { "type": "VM", "id": "b25a5709-f1f8-e942-f0cc-f443eb9b9cf3", "name_label": "SeedBox" }, "id": "1750503696510", "message": "backup VM", "start": 1750503696510, "status": "interrupted", "tasks": [ { "id": "1750503696519", "message": "clean-vm", "start": 1750503696519, "status": "success", "end": 1750503696822, "result": { "merge": false } }, { "id": "1750503697911", "message": "snapshot", "start": 1750503697911, "status": "success", "end": 1750503699564, "result": "6e2edbe9-d4bd-fd23-28b9-db4b03219e96" }, { "data": { "id": "1575a1d8-3f87-4160-94fc-b9695c3684ac", "isFull": false, "type": "remote" }, "id": "1750503699564:0", "message": "export", "start": 1750503699564, "status": "success", "tasks": [ { "id": "1750503701979", "message": "clean-vm", "start": 1750503701979, "status": "success", "end": 1750503702141, "result": { "merge": false } } ], "end": 1750503702142 } ], "warnings": [ { "data": { "attempt": 1, "error": "invalid HTTP header in response body" }, "message": "Retry the VM backup due to an error" } ] }, { "data": { "type": "VM", "id": "3446772a-4110-7a2c-db35-286c73af4ab4", "name_label": "XO" }, "id": "1750503696512", "message": "backup VM", "start": 1750503696512, "status": "interrupted", "tasks": [ { "id": "1750503696518", "message": "clean-vm", "start": 1750503696518, "status": "success", "end": 1750503696693, "result": { "merge": false } }, { "id": "1750503712472", "message": "snapshot", "start": 1750503712472, "status": "success", "end": 1750503713915, "result": "a1bdef52-142c-5996-6a49-169ef390aa2e" }, { "data": { "id": "1575a1d8-3f87-4160-94fc-b9695c3684ac", "isFull": false, "type": "remote" }, "id": "1750503713915:0", "message": "export", "start": 1750503713915, "status": "success", "tasks": [ { "id": "1750503716280", "message": "clean-vm", "start": 1750503716280, "status": "success", "end": 1750503716383, "result": { "merge": false } } ], "end": 1750503716385 } ], "warnings": [ { "data": { "attempt": 1, "error": "invalid HTTP header in response body" }, "message": "Retry the VM backup due to an error" } ] }, { "data": { "type": "VM", "id": "bce2b7f4-d602-5cdf-b275-da9554be61d3", "name_label": "iVentoy" }, "id": "1750503702145", "message": "backup VM", "start": 1750503702145, "status": "interrupted", "tasks": [ { "id": "1750503702148", "message": "clean-vm", "start": 1750503702148, "status": "success", "end": 1750503702233, "result": { "merge": false } }, { "id": "1750503702532", "message": "snapshot", "start": 1750503702532, "status": "success", "end": 1750503704850, "result": "05c5365e-3bc5-4640-9b29-0684ffe6d601" }, { "data": { "id": "1575a1d8-3f87-4160-94fc-b9695c3684ac", "isFull": false, "type": "remote" }, "id": "1750503704850:0", "message": "export", "start": 1750503704850, "status": "interrupted", "tasks": [ { "id": "1750503706813", "message": "transfer", "start": 1750503706813, "status": "interrupted" } ] } ], "infos": [ { "message": "Transfer data using NBD" } ] }, { "data": { "type": "VM", "id": "e0a3093a-52fd-f8dc-1c39-075eeb9d0314", "name_label": "Docker of Things" }, "id": "1750503716389", "message": "backup VM", "start": 1750503716389, "status": "interrupted", "tasks": [ { "id": "1750503716395", "message": "clean-vm", "start": 1750503716395, "status": "success", "warnings": [ { "data": { "path": "/xo-vm-backups/e0a3093a-52fd-f8dc-1c39-075eeb9d0314/20250604T160135Z.json", "actual": 6064872448, "expected": 6064872960 }, "message": "cleanVm: incorrect backup size in metadata" } ], "end": 1750503716886, "result": { "merge": false } }, { "id": "1750503717182", "message": "snapshot", "start": 1750503717182, "status": "success", "end": 1750503719640, "result": "9effb56d-68e6-8015-6bd5-64fa65acbada" }, { "data": { "id": "1575a1d8-3f87-4160-94fc-b9695c3684ac", "isFull": false, "type": "remote" }, "id": "1750503719640:0", "message": "export", "start": 1750503719640, "status": "interrupted", "tasks": [ { "id": "1750503721601", "message": "transfer", "start": 1750503721601, "status": "interrupted" } ] } ], "infos": [ { "message": "Transfer data using NBD" } ] } ], "end": 1750504870213, "result": { "message": "worker exited with code null and signal SIGTERM", "name": "Error", "stack": "Error: worker exited with code null and signal SIGTERM\n at ChildProcess.<anonymous> (file:///opt/xo/xo-builds/xen-orchestra-202506202218/@xen-orchestra/backups/runBackupWorker.mjs:24:48)\n at ChildProcess.emit (node:events:518:28)\n at ChildProcess.patchedEmit [as emit] (/opt/xo/xo-builds/xen-orchestra-202506202218/@xen-orchestra/log/configure.js:52:17)\n at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)\n at Process.callbackTrampoline (node:internal/async_hooks:130:17)" } } [image: 1750514167267-screenshot-2025-06-21-095553.png]
  • Everything related to Xen Orchestra's REST API

    63 Topics
    486 Posts
    olivierlambertO
    This page is being improved as we speak It shouldn't be in "Future improvements" because it's here
  • Terraform, Packer or any tool to do IaC

    32 Topics
    295 Posts
    CyrilleC
    Hello there, We release a new version Terraform provider with improvements of the VM disk lifecycle! Now you can expand a VM disk with Terraform without data loss. Read the release note: https://github.com/vatesfr/terraform-provider-xenorchestra/releases/tag/v0.32.0
  • This topic is deleted!

    1
    1
    0 Votes
    1 Posts
    43 Views
    No one has replied
  • Help with stuck delta backups.

    9
    0 Votes
    9 Posts
    1k Views
    olivierlambertO
    You have a problem on your XCP-ng, this is not related to a XO issue. It's stuck doing a snapshot, which is really a basic operation that shouldn't fail at all. Double check your SMlog on the host.
  • Remotes not mounted after upgrade

    4
    0 Votes
    4 Posts
    709 Views
    2
    It looks like I can remove the remotes and set them up again, and configure the backup jobs to address the new share. That's okay, there are not too many.
  • How to start an XOA continuous replication image at the backup end?

    7
    0 Votes
    7 Posts
    747 Views
    olivierlambertO
    You need one XOA, connected to both pools. It will stream the VM content (delta) from the source to the destination. You don't need any directly link between your SR in both DCs. You DCs can be isolated with just one XOA connected to both, and it will work.
  • This topic is deleted!

    1
    0 Votes
    1 Posts
    3 Views
    No one has replied
  • Can't access Orchestra after build

    4
    1 Votes
    4 Posts
    560 Views
    B
    I was able to fix my issue by adding http to firewalld.
  • Delta backup failes if previous transfer isnt done.

    4
    1
    0 Votes
    4 Posts
    388 Views
    olivierlambertO
    No XCP-ng/XenServer workaround, it's a "built-in"/by design limitation (it's considered as "stalled"). So your goal would be to avoid these extremely long tasks (ie: to finish before the task is destroyed)
  • scheldued snapshots and manual snapshots

    Moved
    6
    1
    0 Votes
    6 Posts
    971 Views
    olivierlambertO
    You'll end up with a chain, yes (so every read will need to make "hops" on each member of the chain). Longer the chain, slower the operation. But it shouldn't be too catastrophic either (for a heavily IOPS loaded VM, you'll notice the difference, but not in other less IOPS hungry scenarios)
  • Changing delta backup schedules breaks backups.

    5
    0 Votes
    5 Posts
    312 Views
    ForzaF
    The error I get when changing snapshot schedule is [image: 1554793685216-delta-fail_1.png]
  • shanshot scheldue - no snapshots visible in XCP-center

    Moved
    5
    1
    0 Votes
    5 Posts
    510 Views
    akurzawaA
    Sure, thanks. I'm using xoa and desktop app simultaneously. I hope that the desktop app will be available for long time.
  • Delete Log Backup-NG

    2
    1
    0 Votes
    2 Posts
    722 Views
    olivierlambertO
    No, because those logs are useful in case of problem (to find a potential bug). However, they are garbage collected automatically (the oldest ones) after a while.
  • Unable to copy a machine on the same host.

    9
    1
    0 Votes
    9 Posts
    2k Views
    olivierlambertO
    All I can tell is that's not an usual issue. Never heard of problem of this kind before… So yeah, there is something weird. Could be related to NFS permissions maybe, or something between XCP-ng and your NFS host. It's not XCP-ng code directly, because otherwise everyone else using NFS will endure the same problems. So it's more a "combo" between XCP-ng and your NFS within your infrastructure. Please read the wiki on where to check logs (especially SMlog) to spot potential hints on where's the issue.
  • Limit backup speeds

    6
    0 Votes
    6 Posts
    942 Views
    nikadeN
    @S-Pam Having a VIF for management and one for storage-traffic is a good solution, we do it that way to avoid crippling management during backups.
  • Where are the logs for "Backup NG" kept?

    Moved
    3
    1
    0 Votes
    3 Posts
    529 Views
    D
    sorry mate - you are correct - will do
  • XO new boot delay feature not working

    Moved
    8
    0 Votes
    8 Posts
    1k Views
    P
    Any ideas for a more elegant way than to write a script? For my point of view vApps should work without HA mode but unfortunately it doesn't work on my XCP-ng server. Only console/script start of the vApp using xe appliance-start uuid= "set here the uuid of the vAppp" is working. But it is again rework for the next XCP-ng version, I think manually set vices in systemd are not persistent through an update. Will vApp supported also in future? there are some discussions about it, if really needed.
  • Hardware for Orchestra

    Moved
    15
    0 Votes
    15 Posts
    4k Views
    A
    sorry for that okay will change it
  • xoa 5.30, cannot restart the host after 12 updates applied

    Moved
    2
    2
    0 Votes
    2 Posts
    245 Views
    olivierlambertO
    A soft reboot will try to migrate existing VMs (ie running on this host) to another host in the pool. Using Xen motion, so only if there is a shared storage. If it fails, then you need a force reboot, that will shutdown the VM, and then reboot the host.
  • Auto Power On VMs when reboot host

    Moved
    10
    0 Votes
    10 Posts
    8k Views
    olivierlambertO
    IIRC, we based the button in XO to do both modification at the same time when you set it up for a VM (in the pool if it's not enabled and in the VM). However, it's been a while I didn't check the code, I'll take a look to be sure
  • How do you remotly manage XO

    Moved
    8
    0 Votes
    8 Posts
    2k Views
    akurzawaA
    Only thru VPN.
  • Email alerts

    Moved
    3
    0 Votes
    3 Posts
    2k Views
    akurzawaA
    Thank you