@heman Yep, I have a separate storage network from my management network. Thanks for the tip! I will check it out...

@danp Thanks. I rebooted a couple of times and the issue persisted.

Where are the logs?

@danp Just upgraded today from 7bacd781c to 2096c782e

[info] Updating xen-orchestra from '7bacd781c' to '2096c782e'

Greetings all,

Running XO community edition (built from sources) connected to my XCP-NG 8.2 servers. I enabled the Usage Reporting plugin a couple of days ago, and now I have over 220 stuck tasks that I can't disable or remove. It appears my recent backups are failing as well (presumably due to the number of stuck backups).

---

Any idea how to find out why the tasks are stuck and how can I delete/unfreeze the tasks?

@thenorthernlight Sorry for all the back-and-forth questions, but I have been trying to reproduce this on my test machine.

Right now, I have a test machine (VM) running with 3 NICs - eth0, eth1, and eth2. I have assigned eth2 as my management interface with eth0/eth1 inside a bond (active/standby). Next, I configured a VLAN with IP.

I deleted the bond via Home-->Host --> Network, and it seemed to get deleted. However, the VLAN assigned to the bond still remained. Odd. At any rate, I was able to delete the bond.

That said, how are you trying to delete the bond? From Home --> Pools or from Home -->Hosts?

@thenorthernlight Is your management interface on a separate physical NIC?

@thenorthernlight Do you have any VLANs or IPs attached to the bond?

@olivierlambert Thanks!

@olivierlambert AH! Thanks, I missed that before.

What if the XO config is corrupt. Is there any way to only export the backup schedule?

Looking for help to backup our scheduled jobs in XO. I see an option to backup VMs and Metadata, but no option to backup the backup jobs.

Background
I had to reset our XO configuration today (community edition - built from sources) using this link. Upon logging in, I noticed the backup jobs were lost (as expected). Once I re-added the backup jobs, I could not find a way to backup the backup jobs. The reset was necessary because an old SR Remote was unreachable, and I could not remove it in XO. This old/unreachable SR was causing issues with restoring VMs and metadata (UI would spin and hang)

Thanks for any pointers.

@stormi @olivierlambert @beshleman
FYI - I have opened a bug in github to track this issue

Greetings all,

Per this thread, I ran into the same problem: Delta backups with a UEFI bios won't boot. Instead of adding to that thread, I felt it would be better to start a new thread with the "bug" info in the title. Copying @creoleMalady and @olivierlambert as they were part of the initial thread.

Problem
When Delta backups are taken, the UEFI boot environment variables are not preserved from the full copy. It appears a blank/clean UEFI bios is added to the restore process to get the VM booted. When this happens, the boot list presented to the VM does not include the installed OS. The current workarounds are (1) boot from a rescue disk and reinstall grub, or (2) manually boot the system via the UEFI Shell then add the proper boot disk to the UEFI boot list.

Fix
The fix is to ensure the UEFI boot environment parameters are preserved from the running VM into the Delta copy. This is presumably inside the delta backup code.

---

Below is the UEFI boot list on a non-working system:

root@Debian-11-delta-restore:~# efibootmgr                                           
BootCurrent: 0003
Timeout: 0 seconds
BootOrder: 0001,0000,0003,0002
Boot0000* UiApp
Boot0001* UEFI Misc Device
Boot0002* UEFI PXEv4 (MAC:0EE793689990)
Boot0003* EFI Internal Shell
                                                    <-- No local boot devices defined.         

---

Below is the UEFI boot list on a working system (note the last entry "Boot0004* debian"):

root@Debian-11-full-restore:~# efibootmgr                                           
BootCurrent: 0005
Timeout: 0 seconds
BootOrder: 0005,0001,0000,0004,0003
Boot0000* UiApp
Boot0001* UEFI Misc Device
Boot0002* UEFI PXEv4 (MAC:1A5848A7B5DD)
Boot0003* EFI Internal Shell
Boot0004* debian.                                    <-- UEFI boot param kept from full backup 

---

Workaround 1: Full grub reinstall

• Mount a rescue disk to the VM
• Boot into rescue mode
• Select correct OS disk
• Reinstall grub

---

Workaround 2: Boot via EFI Shell then re-install grub

• At the UEFI "Shell" prompt, type map to get a list of EFI partitions. On my system, FS0: is the EFI partition
• Type ls FS0:\EFI to get a list of installed OSs. On my system, debian is the name of the installed OS
• Type ls FS0:\EFI\<os> to get a list of grub efi binaries. On my system, grubx64.efi is the bootable efi file
• Run the efi binary. On my system, I used FS0:\EFI\debian\grubx64.efi to get my system running
• Once the OS is running, reinstall grub.

Note: the UEFI Shell expects standard "DOS" partition identifiers. Specifically, note the use of drive:backslash (fs0:\) vs standard unix path/forwardslash (/fs0/)

For additional details on fixing UEFI grub installs, visit this page. It has much more detail on repairing UEFI boot problems.

@thenorthernlight said in New XOA setup, cant register:

@rkelley I already have the compiled XO version working without issue. However we are moving to paid support, so I'm setting up XOA in preparation. This is an entirely new install using the Quick Deploy, so its rather odd that I cant do anything else.

Gotcha - makes sense. Hope you get your issues sorted out.

@danp LOL, no worries dude! I know you are here to help people, and I (for one) really appreciate all the feedback and guidance you give. Hope I didn't come across too argumentative.

@danp said in New XOA setup, cant register:

@rkelley You offered a method of getting XO up and running.

FWIW, XO is not the same thing as XOA, so that wouldn't solve an issue with not being able to register XOA.

Yes, I know. But, aside from the registration part, don't they do pretty much the exact same thing? My point is, @TheNorthernLight might not know XOA is a paid product (requires subscription) when he can get XO (built from sources) free.

In the end, he is just trying to get the Orchestra server up and running. He can do it via XOA or XO from sources. Unless I totally misunderstand the relationship between the two...

@danp Yup. I was offering another method to get his XOA running (to see if he would run into the same issue).

@thenorthernlight Maybe you can try running an automated XOA installer? Here on GitHub

@danp Yes, just rebooted XOA. Problem still exists.

@darkbeldin Here are the steps I used:

• Install 3 node cluster with XCP-NG 8.2
• Install XOA
• Reconfigure mgmt interface - lose all access to XCP-NG cluster.
• Try to force-remove (eject) nodes from cluster. Unsuccessful.
• Remove nodes from XOA
• Reinstall XCP-NG on nodes - without HA (nodes are standalone)
• Add node-1 to XOA; good
• Add node-2 to XOA - Error: This pool is already connected
• Add node-3 to XOA - Error: This pool is already connected

It seems XOA is not cleaning up some internal data structures if it sees the new standalone nodes as part of an old cluster. This seems like a bug to me.

@danp I am getting this message when adding more standalone servers to XO. These servers used to be in a cluster before the cluster was messed up.

Thanks for the pointers on cleaning up XO. Before I do that, I want @olivierlambert to have a chance to review the log files in case I have run into a bug with XO.