Restore metadata and RESTORE_TARGET_MISSING_DEVICE error

DepussayIT

Hello,

I'm testing restoring metadata from an old POOL HS to a new hypervisor.

After moving metadata backup from old POOL HS to metadata backup folder of new hypervisor, I get "RESTORE_TARGET_MISSING_DEVICE(eth3)" error.

Indeed the new hypervisor has 1 ethernet port less than the old master POOL HS.

I know I can get around this error by adding an ethernet port on the new hypervisor, but that may be an additional strain.

I don't understand why the Ethernet port difference is a problem? Do you know the reason?
(in addition, once the metadata has been restored, you have to go through the "network reconfiguration" box to properly recover the new hypervisor)

jaucremann

I'm having this same issue. Were you ever able to find a solution or does anyone have a solution? Thanks.

MTTUPM

Hi,

I ran into the same problem when testing for disaster recovery.

One possible solution I found when "ethX" is missing: set up an XCP-NG host as guest (nested virtualization) and assign it enough NICs. Then restore on the virtualized XCP-NG. This will at least allow you to restore.
This however did not work for: In my testing environment the "dead" host (only host in pool) was a Dell with iDrac-passthrough enabled. The NIC's name was 'idrac' on that host.
Restore seems to be name-sensitive for the NICs. I did not find a way to create a NIC named 'idrac' for a virtualized XCP-NG.

This behavior is described in this document, section "Physical Machine failure - To restore a pool with all hosts failed" with no solution given.

olivierlambert

Metadata restore is meant to restore on a similar host. Alternatively, restore the real VM backups via XO.

I also wonder if there's a way to just partially restore metadata, but it might be complex if you have missing NICs, therefore you'll have missing networks.

MTTUPM

I did restore the VMs from their backups (full backups, delta backups, online snapshots, offline snapshots, XOA, XO self compiled) - it worked flawlessly and it is very much good enough for me.

My post was just meant as a small contribution to a possible future solution to the problem. Larger environments might be interested in disaster-recovering metadata on non-identical hardware.

If there was some way to assign arbitrary names to virtual NICs, the "restore into a virtualized XCP-NG" approach would most likely work. Maybe this is a simple thing for you to implement - and thereby offer an even better DR solution to larger installations.

olivierlambert

@MTTUPM said in Restore metadata and RESTORE_TARGET_MISSING_DEVICE error:

Larger environments might be interested in disaster-recovering metadata on non-identical hardware.

That's not a best practice and probably very complex to manage. I'm not saying it shouldn't be done, but we probably have more pressing issues right now. If you find a way to do it with xe CLI, let us know so it's easier to just "expose" in XO

MTTUPM

OK, it can be done. So your lost host/pool had some fancy named network device (let's say idrac). You don't have identical hardware. Solution:

set up a virtualized XCP-NG with the same (more might work) number of NICs as your lost host
access that host's console
xe pif-list # find a NIC you don't need for metadata restore, let's say eth4
xe pif-forget uuid=<that pif>
xe network-list # find the network associated with the forgotten NIC
xe network-destroy uuid=<that network>
ip link set dev eth4 down
ip link set dev eth4 name idrac
ip link set dev idrac up
ip link show # find idrac's mac
xe pif-introduce host-uuid<your virtualized host> mac=<idrac's mac> device=idrac
I think this automatically added a network named idrac to the pool - don't know if it's needed

Now you can add the virtualized host to XO and restore your metadata. Is this a good idea? I don't know. I did not test it any further.

sshughes

We are also performing recovery tests and documenting procedures in case a single host dies. New host has less NICs and our restore is failing with 'RESTORE_TARGET_MISSING_DEVICE(eth1)'. However, it's not that hard to add a NIC, so I did and now eth0, eth1, eth2 all exist but restore still fails with RESTORE_TARGET_MISSING_DEVICE(eth1). Is it also attempting to match MAC addresses? These are small deployments, less than four VMs, Internal Storage.
Usually one Windows Server VM responsible for 90% of SR use, plus a couple of small VMs: XO and other minor functions.
Not crazy about having to restore full VM backups when the SR is fully intact.
Is there a better way?

MTTUPM

@sshughes When you go to 'Home > Hosts > YourRestoreHost > Tab:Network' in XO, is eth1 shown there? If not, it might need introduction.