So another odd thing I'm seeing with this VDI, it's showing the size incorrectly. It shows 180GB of 180GB used up (on thin provisioned SR, both the old and new are), however the VM is only using 140GB of that 180GB. Something definitely went wrong with this VDI during transfer, just not sure what. I will say that I increased the VDI size again and now it displays more accurately, showing 180GB of 185GB used (both in XOA and with vhd-util). Almost behaving as if this was at one point on a thick provisioned SR or something. Just to avoid issues I'm maybe tempted to create a fresh VHD, copy data to that, then delete this one.

@olivierlambert OK perfect, thank you! I've got a S3 Delta test going, will come back and post my results for others to reference once my testing is done.

@ronivay that was it

@olivierlambert thanks for info I can wait for XO 6 version

It's not only this code. But in general, a good way to manage that is: "every message in CAPS is coming from the lower layer" (ie not XO).

@xcprocks said in Restoring a downed host ISNT easy: So, we had a host go down (OS drive failure). No big deal right? According to instructions, just reinstall XCP on a new drive, jump over into XOA and do a metadata restore. Well, not quite. First during installation, you really really must not select any of the disks to create an SR as you could potentially wipe out an SR. Second, you have to do the sr-probe and sr-introduce and pbd-create and pbd-plug to get the SRs back. Third, you then have to use XOA to restore the metadata which according to the directions is pretty simple looking. According to: https://xen-orchestra.com/docs/metadata_backup.html#performing-a-restore "To restore one, simply click the blue restore arrow, choose a backup date to restore, and click OK:" But this isn't quite true. When we did it, the restore threw an error: "message": "no such object d7b6f090-cd68-9dec-2e00-803fc90c3593", "name": "XoError", Panic mode sets in... It can't find the metadata? We try an earlier backup. Same error. We check the backup NFS share--no its there alright. After a couple of hours scouring the internet and not finding anything, it dawns on us... The object XOA is looking for is the OLD server not a backup directory. It is looking for the server that died and no longer exists. The problem is, when you install the new server, it gets a new ID. But the restore program is looking for the ID of the dead server. But how do you tell XOA, to copy the metadata over to the new server? It assumes that you want to restore it over an existing server. It does not provide a drop down list to pick where to deploy it. In an act of desperation, we copied the backup directory to a new location and named it with the ID number of the newly recreated server. Now XOA could restore the metadata and we were able to recover the VMs in the SRs without issue. This long story is really just a way to highlight the need for better host backup in three ways: A) The first idea would be to create better instructions. It ain't nowhere as easy as the documentation says it is and it's easy to mess up the first step so bad that you can wipe out the contents of an SR. The documentation should spell this out. B) The second idea is to add to the metadata backup something that reads the states of SR to PBD mappings and provides/saves a script to restore them. This would ease a lot of the difficulty in the actual restoring of a failed OS after a new OS can be installed. C) The third idea is provide a dropdown during the restoration of the metadata that allows the user to target a particular machine for the restore operation instead of blindly assuming you want to restore it over a machine that is dead and gone. I hope this helps out the next person trying to bring a host back from the dead, and I hope it also helps make XOA a better product. Thanks for a good description of the restore process. I was wary of the metadata-backup option. It sounds simple and good to have, but as you said it is in no way a comprehensive restore of a pool. I'd like to add my own oppinion here. A full pool restore, including network, re-attaching SRs and everything else that is needed to quickly get back up and running. Also a restore pool backup should be available on the boot media. It could look for a NFS/CIFS mount or a USB disk with the backup files on. This would avoid things like issues with bonded networks not working.

@danp Okay that's great. I was just making sure. Thanks for your reply. Jack

We did recently have a failed drive that finished rebuilding a few days ago.

@olivierlambert i'm using a ova import with UEFI comming from virtualbox, folowed all previous option and ddocumentation and still can't get my VM to boot. It's stuck with black screen and no network or whatever informations. Using this packer template to build the VM : { "variables": { "vm_name": "mjolmir-base", "cpu": "2", "ram_size": "2048", "disk_size": "5000", "iso_url": "ubuntu-18.04.5-server-amd64.iso", "iso_checksum_type": "sha1", "iso_checksum": "c5dc820b3b46ccdd94bb1568715852ea3bf772a4", "username" : "vagrant", "password" : "vagrant" }, "builders": [ { "vm_name":"{{user `vm_name`}}", "type": "virtualbox-iso", "format": "ova", "output_directory": "./data", "boot_wait": "5s", "disk_size": "{{user `disk_size`}}", "headless": true, "guest_additions_mode":"disable", "http_directory": "./http/", "iso_url": "{{user `iso_url`}}", "iso_checksum": "{{user `iso_checksum`}}", "communicator":"ssh", "ssh_username": "{{user `username`}}", "ssh_password": "{{user `password`}}", "ssh_timeout": "4h", "shutdown_command": "echo 'vagrant' | sudo -S -E shutdown -P now", "boot_command": [ "e<down><down><down><down><down><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "linux /install/vmlinuz<wait>", " auto<wait>", " console-setup/ask_detect=false<wait>", " console-setup/layoutcode=us<wait>", " console-setup/modelcode=pc105<wait>", " root=/dev/sda", " debconf/frontend=noninteractive<wait>", " debian-installer=fr_FR<wait>", " vga=normal fb=false<wait>", " kbd-chooser/method=fr<wait>", " keyboard-configuration/layout=FR<wait>", " keyboard-configuration/variant=FR<wait>", " locale=fr_FR<wait>", " netcfg/get_domain=localdomain<wait>", " netcfg/get_hostname=mjolmir<wait>", " grub-installer/bootdev=/dev/sda<wait>", " noapic<wait>", " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/http/18.04/preseed-cloud-init.cfg", " -- <wait><enter>", " initrd /install/initrd.gz<wait>", "<f10>" ], "guest_os_type": "Ubuntu_64", "firmware":"efi", "iso_interface": "sata", "vboxmanage": [ [ "modifyvm","{{.Name}}","--memory","{{user `ram_size`}}" ], [ "modifyvm","{{.Name}}","--cpus","{{user `cpu`}}" ], [ "modifyvm","{{.Name}}","--firmware","EFI" ] ] }], "provisioners": [{ "type": "shell", "execute_command": "echo 'vagrant' | sudo -S -E sh {{.Path}}", "scripts": [ "./script/update.sh", "./script/cloud-init.sh", "./script/cleanup.sh" ] }], "post-processors": [ { "keep_input_artifact": true, "output": "../../{{.Provider}}/mjolmir.box", "type": "vagrant" } ] } with provisioning script : #!/bin/sh #Disable conflict service systemctl stop apt-daily.service systemctl kill --kill-who=all apt-daily.service wait until `apt-get updated` has been killed while ! (systemctl list-units --all apt-daily.service | egrep -q '(dead|failed)') do sleep 1; done #Disable service at startup systemctl mask apt-daily.service apt-daily-upgrade.service systemctl disable apt-daily.service apt-daily-upgrade.service systemctl disable apt-daily.timer apt-daily-upgrade.timer # install cloud-init apt-get update && apt-get upgrade apt-get install linux-image-generic xserver-xorg apt-get remove linux-image-generic-hwe-18.04 apt-get install --install-recommends -y cloud-init xe-guest-utilities cloud-initramfs-growroot xserver-xorg-video-all #hostname will be managed by cloud-init, but the current value will not be removed HOSTNAME=`hostname` sed -i "/${HOSTNAME}/d" /etc/hosts sed -i s/enp0s3/eth0/g /etc/network/interfaces sed -i "s/GRUB_CMDLINE_LINUX_DEFAULT=\"quiet\"/GRUB_CMDLINE_LINUX_DEFAULT=\"quiet vga=normal fb=false dsmode\=nocloud\\;s\=\/ratatosk\"/g" /etc/default/grub #sed -i "s/Before\=sysinit\.target/After\=sysinit\.target/g" /etc/systemd/system/cloud-init.target.wants/cloud-init.service #sed -i "s/\[Service\]/\[Service\]\r\nAfter\=\cloud-init.service/g" /lib/systemd/system/xe-daemon.service #sed -i "s/\[Service\]/\[Service\]\r\nExecStartPre\=\/bin\/sleep 30/g" /etc/systemd/system/cloud-init.target.wants/cloud-init.service #sed -i "s/\[Service\]/\[Service\]\r\nExecStartPre\=\/bin\/sleep 30/g" /etc/systemd/system/cloud-init.target.wants/cloud-init-local.service sed -i "s/datasource_list: \[ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, RbxCloud, UpCloud, None \]//g" /etc/cloud/cloud.cfg.d/90_dpkg.cfg cp /tmp/cloud.cfg /etc/cloud/cloud.cfg && rm /tmp/cloud.cfg cat <<EOF >> /etc/cloud/cloud.cfg.d/90_dpkg.cfg datasource_list: [ NoCloud, None ] datasource: NoCloud: seedfrom: /ratatosk/ EOF update-grub mkdir /ratatosk echo "/dev/xvdb /ratatosk vfat defaults 0 2" >> /etc/fstab DI_LOG=stderr /usr/lib/cloud-init/ds-identify --force dpkg-reconfigure -f noninteractive cloud-init stuck on : [image: 1629388720596-034b3235-383a-4aec-8a72-52bc9fb8c9ea-image.png] Any suggestion or whatever i'm missing ? Ubuntu 16.04 OVA created the same way can be imported and booted flawlessly.

On a thin volume, it will only use the "real" space in the VM, so if you are using 40 or 50GiB, yes it will fit on the 78GiB storage

@olivierlambert thank you Olivier!

Both secure boot support and the fix for this KB installation are now available for testing: https://xcp-ng.org/forum/post/41541

FYI, you can always ask for another XOA trial when you need it

It's not that complicated: it's in a folder with the UUID of the backup job, then the VM is within it's UUID folder. That's it.

@zevgeny Hi, You can export the config for your XOA in Settings > Config > Export and import it on your new XOA

@joearnon Can you post logs of both backup instances (of the same job) that ran, see if they are different and in which way? You can also try to reconfigure the back up job to only a single tag parameter to see if that is the issue.

Hi Pechkin000, Try this : npm install url-loader yarn install v1.22.11 [1/5] Validating package.json... [2/5] Resolving packages... [3/5] Fetching packages... info fsevents@2.3.2: The platform "linux" is incompatible with this module. info "fsevents@2.3.2" is an optional dependency and failed compatibility check. Excluding it from installation. info fsevents@1.2.13: The platform "linux" is incompatible with this module. info "fsevents@1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation. [4/5] Linking dependencies... [5/5] Building fresh packages... Done in 22.66s.