XOCE misidentifing Orphaned Snapshots with XCP-ng 8.1

Danp

Is the XS7.1 pool attached to separate NFS pool? And the snapshots existed prior to migration, correct?

Anonabhar

Hi Damp,

Completely different NFS Repos between the two pools. In fact, different storage networks entirely.

I did the migration both ways, first by migrating with the snapshots and also by deleting the snapshots and waiting for the SMGC to finish before doing the migration.

If I delete all snapshots on the machine, wait for SMGC to finish and then create a new snapshot, it gets marked as Orphaned.

Using XenCenter, the snapshots all appear correctly linked to the VM, so I think its just a reporting thing, but I am not sure..

~Peg

olivierlambert

You said using XenCenter it seems fine. What's the other way to create snap you are using and displaying the behavior?

Anonabhar

Sorry, what I meant to say was that if I use XenCenter to examine the VM (going to the snapshot tab) I see the snapshots properly. I also seem them linked properly in XOCE by going to "Home -> VM -> [Pick Any VM}".

Didnt mean to cause confusion..

~Peg

olivierlambert

Ping @badrAZ or @Rajaa-BARHTAOUI who updated the orphaned feature.

If the snapshot disk is correctly attached to a VM snapshot, it shouldn't be displayed there.

badrAZ

Hi @Anonabhar

Thank you for your report.

The XO logic is based on the snapshot_of property which indicates if a snapshot is attached to a VM or not.

After looking at your issue, it means that this property isn't correctly updated on migrating a VM with snapshots to a freshly installed XCP 8.1.

I'm not able to reproduce it on migrating a VM with a snapshot from an old XCP 8.1 to a freshly installed XCP 8.1.

Anonabhar

Hi @badrAZ ,

Thanks for the info. Is there any way I can view what properties are on the snapshot to check this? (and report back)

As a side-note, I have also just taken a snapshot of a VM that was not migrated in and it is also showing the same behavior in the "Dashboard -> Health" screen.

Kind Regards,
~Peg

badrAZ

Yes, you can get the snapshot's record using the xo-cli.

The command is

xo-cli list-objects id=snapshotId

I'll need also the record of its VM. Thank you

Anonabhar

Hi @badrAZ ,

Ok.. I think I got the information you are wanting..

This is the dump from the VM:

root@debian:/home/ken# xo-cli list-objects id=547501f1-51e5-67ea-771d-304ad95145aa
[
{
  "type": "VM",
  "addresses": {
    "0/ipv4/0": "185.192.178.68",
    "0/ipv6/1": "fe80::6c88:c3ff:fe40:be7e",
    "0/ipv6/0": "2a0a:2e40:ac01:802:6c88:c3ff:fe40:be7e"
  },
  "auto_poweron": false,
  "bios_strings": {
    "bios-vendor": "Xen",
    "bios-version": "",
    "system-manufacturer": "Xen",
    "system-product-name": "HVM domU",
    "system-version": "",
    "system-serial-number": "",
    "baseboard-manufacturer": "",
    "baseboard-product-name": "",
    "baseboard-version": "",
    "baseboard-serial-number": "",
    "baseboard-asset-tag": "",
    "baseboard-location-in-chassis": "",
    "enclosure-asset-tag": "",
    "hp-rombios": "",
    "oem-1": "Xen",
    "oem-2": "MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d"
  },
  "blockedOperations": {},
  "boot": {
    "firmware": "bios",
    "order": "cdn"
  },
  "CPUs": {
    "max": 2,
    "number": 2
  },
  "current_operations": {},
  "expNestedHvm": false,
  "mainIpAddress": "185.192.178.68",
  "high_availability": "",
  "memory": {
    "dynamic": [
      2147483648,
      2147483648
    ],
    "static": [
      536870912,
      2147483648
    ],
    "size": 2147483648
  },
  "installTime": null,
  "name_description": "",
  "name_label": "AUC.Debian10-Tester",
  "other": {
    "import_task": "OpaqueRef:66a61f13-3146-4c48-a99c-13ab36bf75c4",
    "instant": "true",
    "base_template_name": "Debian Buster 10",
    "mac_seed": "40154d39-694a-3584-9863-dee48b5a5e68",
    "install-methods": "cdrom,nfs,http,ftp",
    "linux_template": "true"
  },
  "os_version": {
    "name": "Debian GNU/Linux 10 (buster)",
    "uname": "4.19.0-9-amd64",
    "distro": "debian",
    "major": "10",
    "minor": "10"
  },
  "power_state": "Halted",
  "hasVendorDevice": false,
  "snapshots": [
    "c830ce0b-d03d-9cb4-53ce-fbd0bc258208"
  ],
  "startDelay": 0,
  "startTime": null,
  "tags": [],
  "VIFs": [
    "d9fa7453-461e-856f-def5-002d41018251"
  ],
  "virtualizationMode": "hvm",
  "$container": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
  "$VBDs": [
    "6f6390fe-099f-0f15-a985-6da6cb659824",
    "58505004-b392-c293-4a46-9b797fee4b92"
  ],
  "VGPUs": [],
  "$VGPUs": [],
  "vga": "std",
  "videoram": "8",
  "coresPerSocket": 2,
  "id": "547501f1-51e5-67ea-771d-304ad95145aa",
  "uuid": "547501f1-51e5-67ea-771d-304ad95145aa",
  "$pool": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
  "$poolId": "e78d04fc-b33f-6026-28bb-cdceb8e44863"
}
]

This is the dump from the snapshot:

root@debian:/home/ken# xo-cli list-objects id=c830ce0b-d03d-9cb4-53ce-fbd0bc258208
[
{
  "type": "VM-snapshot",
  "addresses": {
    "0/ipv4/0": "185.192.178.68",
    "0/ipv6/1": "fe80::6c88:c3ff:fe40:be7e",
    "0/ipv6/0": "2a0a:2e40:ac01:802:6c88:c3ff:fe40:be7e"
  },
  "auto_poweron": false,
  "bios_strings": {
    "bios-vendor": "Xen",
    "bios-version": "",
    "system-manufacturer": "Xen",
    "system-product-name": "HVM domU",
    "system-version": "",
    "system-serial-number": "",
    "baseboard-manufacturer": "",
    "baseboard-product-name": "",
    "baseboard-version": "",
    "baseboard-serial-number": "",
    "baseboard-asset-tag": "",
    "baseboard-location-in-chassis": "",
    "enclosure-asset-tag": "",
    "hp-rombios": "",
    "oem-1": "Xen",
    "oem-2": "MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d"
  },
  "blockedOperations": {},
  "boot": {
    "firmware": "bios",
    "order": "cdn"
  },
  "CPUs": {
    "max": 2,
    "number": 2
  },
  "current_operations": {},
  "expNestedHvm": false,
  "mainIpAddress": "185.192.178.68",
  "high_availability": "",
  "memory": {
    "dynamic": [
      2147483648,
      2147483648
    ],
    "static": [
      536870912,
      2147483648
    ],
    "size": 2147483648
  },
  "installTime": null,
  "name_description": "",
  "name_label": "Test snapshot",
  "other": {
    "import_task": "OpaqueRef:66a61f13-3146-4c48-a99c-13ab36bf75c4",
    "instant": "true",
    "base_template_name": "Debian Buster 10",
    "mac_seed": "40154d39-694a-3584-9863-dee48b5a5e68",
    "install-methods": "cdrom,nfs,http,ftp",
    "linux_template": "true"
  },
  "os_version": {
    "name": "Debian GNU/Linux 10 (buster)",
    "uname": "4.19.0-9-amd64",
    "distro": "debian",
    "major": "10",
    "minor": "10"
  },
  "power_state": "Halted",
  "hasVendorDevice": false,
  "snapshots": [],
  "startDelay": 0,
  "startTime": null,
  "tags": [],
  "VIFs": [
    "0dde6812-79f8-327d-e69c-e098b6fec730"
  ],
  "virtualizationMode": "hvm",
  "$container": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
  "$VBDs": [
    "a2f64b99-6e5d-3cea-8331-fea0c91c0b8e",
    "f95fc772-a9fb-e944-8f7b-4aefc4d0124d"
  ],
  "VGPUs": [],
  "$VGPUs": [],
  "vga": "std",
  "videoram": "8",
  "coresPerSocket": 2,
  "snapshot_time": 1599054372,
  "id": "c830ce0b-d03d-9cb4-53ce-fbd0bc258208",
  "uuid": "c830ce0b-d03d-9cb4-53ce-fbd0bc258208",
  "$pool": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
  "$poolId": "e78d04fc-b33f-6026-28bb-cdceb8e44863"
}
]

If there is anything else I can give you, please let me know..

Kind Regards,
~Peg

badrAZ

Thank you for your response,

The field snapshot_of doesn't exist. Please give me the output of

xe vm-list uuid=snapshotUuid params=snapshot-of

Thank you

Anonabhar

Hi @badrAZ

Thanks for your help. I got a really strange response (as in... Nothing...)

[09:18 shineyg8-2-1 ~]# xe vm-list uuid=c830ce0b-d03d-9cb4-53ce-fbd0bc258208 params=snapshot-of
[09:19 shineyg8-2-1 ~]#

But if I do it against the original (base) VM I get:

[09:17 shineyg8-2-1 ~]# xe vm-list uuid=547501f1-51e5-67ea-771d-304ad95145aa params=snapshot-of
snapshot-of ( RO)    : <not in database>

olivierlambert

That's because you need to use xe snapshot-param-list.

Anonabhar

@olivierlambert & @badrAZ

Ah, that is better 8-) This is what I get this time...

[09:56 shineyg8-2-1 ~]# xe snapshot-param-list uuid=c830ce0b-d03d-9cb4-53ce-fbd0bc258208 
uuid ( RO)                                  : c830ce0b-d03d-9cb4-53ce-fbd0bc258208
                            name-label ( RW): Test snapshot
                      name-description ( RW): 
                          user-version ( RW): 1
                         is-a-template ( RW): true
                   is-default-template ( RW): false
                         is-a-snapshot ( RO): true
                           snapshot-of ( RO): 547501f1-51e5-67ea-771d-304ad95145aa
                             snapshots ( RO): 
                         snapshot-time ( RO): 20200902T13:46:12Z
                         snapshot-info ( RO): disk-snapshot-type: crash_consistent; power-state-at-snapshot: Halted
                                parent ( RO): <not in database>
                              children ( RO): 547501f1-51e5-67ea-771d-304ad95145aa
                     is-control-domain ( RO): false
                           power-state ( RO): halted
                         memory-actual ( RO): 0
                         memory-target ( RO): 0
                       memory-overhead ( RO): 20971520
                     memory-static-max ( RW): 2147483648
                    memory-dynamic-max ( RW): 2147483648
                    memory-dynamic-min ( RW): 2147483648
                     memory-static-min ( RW): 536870912
                      suspend-VDI-uuid ( RW): <not in database>
                       suspend-SR-uuid ( RW): <not in database>
                          VCPUs-params (MRW): 
                             VCPUs-max ( RW): 2
                      VCPUs-at-startup ( RW): 2
                actions-after-shutdown ( RW): Destroy
                  actions-after-reboot ( RW): Restart
                   actions-after-crash ( RW): Restart
                         console-uuids (SRO): 
                                   hvm ( RO): false
                              platform (MRW): timeoffset: 1; videoram: 8; hpet: true; secureboot: false; device-model: qemu-upstream-compat; apic: true; device_id: 0001; vga: std; nx: true; pae: true; viridian: false; acpi: 1; cores-per-socket: 2
                    allowed-operations (SRO): destroy; export; revert; clone; copy
                    current-operations (SRO): 
                    blocked-operations (MRW): 
                   allowed-VBD-devices (SRO): 1; 2; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254
                   allowed-VIF-devices (SRO): 1; 2; 3; 4; 5; 6
                        possible-hosts ( RO): ec347d65-341f-4ad2-941a-2e8544539ae7; 64e44a18-ca98-4014-9265-dc0854ffa6cd; 74d1a6d6-0660-4d4f-825d-60d609fc4fec; d4fb12f2-8d30-4efd-bf27-40006f6ec809; 92780235-d330-470f-88dd-b62d755ef119
                           domain-type ( RW): hvm
                   current-domain-type ( RO): unspecified
                       HVM-boot-policy ( RW): BIOS order
                       HVM-boot-params (MRW): firmware: bios; order: cdn
                 HVM-shadow-multiplier ( RW): 1.000
                             PV-kernel ( RW): 
                            PV-ramdisk ( RW): 
                               PV-args ( RW): 
                        PV-legacy-args ( RW): 
                         PV-bootloader ( RW): 
                    PV-bootloader-args ( RW): 
                   last-boot-CPU-flags ( RO): vendor: GenuineIntel; features: 1fcbfbff-97ba2223-2d93fbff-00000403-00000001-00000000-00000000-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000
                      last-boot-record ( RO): ''
                           resident-on ( RO): <not in database>
                              affinity ( RW): <not in database>
                          other-config (MRW): import_task: OpaqueRef:66a61f13-3146-4c48-a99c-13ab36bf75c4; instant: true; base_template_name: Debian Buster 10; mac_seed: 40154d39-694a-3584-9863-dee48b5a5e68; install-methods: cdrom,nfs,http,ftp; linux_template: true
                                dom-id ( RO): -1
                       recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="32"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions>
                         xenstore-data (MRW): vm-data: ; vm-data/mmio-hole-size: 268435456
            ha-always-run ( RW) [DEPRECATED]: false
                   ha-restart-priority ( RW): 
                                 blobs ( RO): 
                            start-time ( RO): 19700101T00:00:00Z
                          install-time ( RO): 19700101T00:00:00Z
                          VCPUs-number ( RO): 0
                     VCPUs-utilisation (MRO): 
                            os-version (MRO): name: Debian GNU/Linux 10 (buster); uname: 4.19.0-9-amd64; distro: debian; major: 10; minor: 10
                    PV-drivers-version (MRO): major: 8; minor: 0; micro: 50; build: 1
    PV-drivers-up-to-date ( RO) [DEPRECATED]: true
                                memory (MRO): 
                                 disks (MRO): 
                                  VBDs (SRO): a2f64b99-6e5d-3cea-8331-fea0c91c0b8e; f95fc772-a9fb-e944-8f7b-4aefc4d0124d
                              networks (MRO): 0/ip: 185.192.178.68; 0/ipv4/0: 185.192.178.68; 0/ipv6/1: fe80::6c88:c3ff:fe40:be7e; 0/ipv6/0: 2a0a:2e40:ac01:802:6c88:c3ff:fe40:be7e
                   PV-drivers-detected ( RO): true
                                 other (MRO): platform-feature-multiprocessor-suspend: 1; has-vendor-device: 0; feature-suspend: 1; feature-poweroff: 1; feature-reboot: 1; feature-vcpu-hotplug: 1; feature-balloon: 1; shutdown: 
                                  live ( RO): true
            guest-metrics-last-updated ( RO): 20200823T20:32:37Z
                   can-use-hotplug-vbd ( RO): unspecified
                   can-use-hotplug-vif ( RO): unspecified
              cooperative ( RO) [DEPRECATED]: true
                                  tags (SRW): 
                             appliance ( RW): <not in database>
                     snapshot-schedule ( RW): <not in database>
                      is-vmss-snapshot ( RO): false
                           start-delay ( RW): 0
                        shutdown-delay ( RW): 0
                                 order ( RW): 0
                               version ( RO): 0
                         generation-id ( RO): 
             hardware-platform-version ( RO): 0
                     has-vendor-device ( RW): false
                       requires-reboot ( RO): false
                       reference-label ( RO): debian-10
                          bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; baseboard-manufacturer: ; baseboard-product-name: ; baseboard-version: ; baseboard-serial-number: ; baseboard-asset-tag: ; baseboard-location-in-chassis: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d


[09:56 shineyg8-2-1 ~]#

badrAZ

@Anonabhar It seems that the property snapshot-of isn't correctly reported by XO. Did you tried to disconnect/reconnect the server containing this snapshot to see if it resolves your issue?

And please give me the version of your xo-server.

Anonabhar

Hi @badrAZ

Well.. Monkey Poo!!!!!

Disconnecting the server and re-enabling the server did clear the issue. I have made a few snapshots and run a few backups to cause snapshots to be rotated and all is looking good..

Something must have been stuck in the original connection?

FYI:

Please mark this as solved...

~Peg

olivierlambert

Which isn't normal is the fact XO should "reset" the connection and fetch all objects again to avoid "partial" object info… @badrAZ we should investigate on that, we can't rely on just disconnect/reconnect in such situations.

badrAZ

@olivierlambert I've created the dedicated Github issue