XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XOCE misidentifing Orphaned Snapshots with XCP-ng 8.1

    Scheduled Pinned Locked Moved Solved Xen Orchestra
    18 Posts 4 Posters 1.4k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • badrAZB Offline
      badrAZ
      last edited by badrAZ

      Hi @Anonabhar

      Thank you for your report.

      The XO logic is based on the snapshot_of property which indicates if a snapshot is attached to a VM or not.

      After looking at your issue, it means that this property isn't correctly updated on migrating a VM with snapshots to a freshly installed XCP 8.1.

      I'm not able to reproduce it on migrating a VM with a snapshot from an old XCP 8.1 to a freshly installed XCP 8.1.

      AnonabharA 1 Reply Last reply Reply Quote 0
      • AnonabharA Offline
        Anonabhar @badrAZ
        last edited by

        Hi @badrAZ ,

        Thanks for the info. Is there any way I can view what properties are on the snapshot to check this? (and report back)

        As a side-note, I have also just taken a snapshot of a VM that was not migrated in and it is also showing the same behavior in the "Dashboard -> Health" screen.

        Kind Regards,
        ~Peg

        1 Reply Last reply Reply Quote 0
        • badrAZB Offline
          badrAZ
          last edited by

          Yes, you can get the snapshot's record using the xo-cli.

          The command is

          xo-cli list-objects id=snapshotId

          I'll need also the record of its VM. Thank you πŸ™‚

          AnonabharA 1 Reply Last reply Reply Quote 0
          • AnonabharA Offline
            Anonabhar @badrAZ
            last edited by

            Hi @badrAZ ,

            Ok.. I think I got the information you are wanting..

            This is the dump from the VM:

            root@debian:/home/ken# xo-cli list-objects id=547501f1-51e5-67ea-771d-304ad95145aa
            [
            {
              "type": "VM",
              "addresses": {
                "0/ipv4/0": "185.192.178.68",
                "0/ipv6/1": "fe80::6c88:c3ff:fe40:be7e",
                "0/ipv6/0": "2a0a:2e40:ac01:802:6c88:c3ff:fe40:be7e"
              },
              "auto_poweron": false,
              "bios_strings": {
                "bios-vendor": "Xen",
                "bios-version": "",
                "system-manufacturer": "Xen",
                "system-product-name": "HVM domU",
                "system-version": "",
                "system-serial-number": "",
                "baseboard-manufacturer": "",
                "baseboard-product-name": "",
                "baseboard-version": "",
                "baseboard-serial-number": "",
                "baseboard-asset-tag": "",
                "baseboard-location-in-chassis": "",
                "enclosure-asset-tag": "",
                "hp-rombios": "",
                "oem-1": "Xen",
                "oem-2": "MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d"
              },
              "blockedOperations": {},
              "boot": {
                "firmware": "bios",
                "order": "cdn"
              },
              "CPUs": {
                "max": 2,
                "number": 2
              },
              "current_operations": {},
              "expNestedHvm": false,
              "mainIpAddress": "185.192.178.68",
              "high_availability": "",
              "memory": {
                "dynamic": [
                  2147483648,
                  2147483648
                ],
                "static": [
                  536870912,
                  2147483648
                ],
                "size": 2147483648
              },
              "installTime": null,
              "name_description": "",
              "name_label": "AUC.Debian10-Tester",
              "other": {
                "import_task": "OpaqueRef:66a61f13-3146-4c48-a99c-13ab36bf75c4",
                "instant": "true",
                "base_template_name": "Debian Buster 10",
                "mac_seed": "40154d39-694a-3584-9863-dee48b5a5e68",
                "install-methods": "cdrom,nfs,http,ftp",
                "linux_template": "true"
              },
              "os_version": {
                "name": "Debian GNU/Linux 10 (buster)",
                "uname": "4.19.0-9-amd64",
                "distro": "debian",
                "major": "10",
                "minor": "10"
              },
              "power_state": "Halted",
              "hasVendorDevice": false,
              "snapshots": [
                "c830ce0b-d03d-9cb4-53ce-fbd0bc258208"
              ],
              "startDelay": 0,
              "startTime": null,
              "tags": [],
              "VIFs": [
                "d9fa7453-461e-856f-def5-002d41018251"
              ],
              "virtualizationMode": "hvm",
              "$container": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
              "$VBDs": [
                "6f6390fe-099f-0f15-a985-6da6cb659824",
                "58505004-b392-c293-4a46-9b797fee4b92"
              ],
              "VGPUs": [],
              "$VGPUs": [],
              "vga": "std",
              "videoram": "8",
              "coresPerSocket": 2,
              "id": "547501f1-51e5-67ea-771d-304ad95145aa",
              "uuid": "547501f1-51e5-67ea-771d-304ad95145aa",
              "$pool": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
              "$poolId": "e78d04fc-b33f-6026-28bb-cdceb8e44863"
            }
            ]
            

            This is the dump from the snapshot:

            root@debian:/home/ken# xo-cli list-objects id=c830ce0b-d03d-9cb4-53ce-fbd0bc258208
            [
            {
              "type": "VM-snapshot",
              "addresses": {
                "0/ipv4/0": "185.192.178.68",
                "0/ipv6/1": "fe80::6c88:c3ff:fe40:be7e",
                "0/ipv6/0": "2a0a:2e40:ac01:802:6c88:c3ff:fe40:be7e"
              },
              "auto_poweron": false,
              "bios_strings": {
                "bios-vendor": "Xen",
                "bios-version": "",
                "system-manufacturer": "Xen",
                "system-product-name": "HVM domU",
                "system-version": "",
                "system-serial-number": "",
                "baseboard-manufacturer": "",
                "baseboard-product-name": "",
                "baseboard-version": "",
                "baseboard-serial-number": "",
                "baseboard-asset-tag": "",
                "baseboard-location-in-chassis": "",
                "enclosure-asset-tag": "",
                "hp-rombios": "",
                "oem-1": "Xen",
                "oem-2": "MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d"
              },
              "blockedOperations": {},
              "boot": {
                "firmware": "bios",
                "order": "cdn"
              },
              "CPUs": {
                "max": 2,
                "number": 2
              },
              "current_operations": {},
              "expNestedHvm": false,
              "mainIpAddress": "185.192.178.68",
              "high_availability": "",
              "memory": {
                "dynamic": [
                  2147483648,
                  2147483648
                ],
                "static": [
                  536870912,
                  2147483648
                ],
                "size": 2147483648
              },
              "installTime": null,
              "name_description": "",
              "name_label": "Test snapshot",
              "other": {
                "import_task": "OpaqueRef:66a61f13-3146-4c48-a99c-13ab36bf75c4",
                "instant": "true",
                "base_template_name": "Debian Buster 10",
                "mac_seed": "40154d39-694a-3584-9863-dee48b5a5e68",
                "install-methods": "cdrom,nfs,http,ftp",
                "linux_template": "true"
              },
              "os_version": {
                "name": "Debian GNU/Linux 10 (buster)",
                "uname": "4.19.0-9-amd64",
                "distro": "debian",
                "major": "10",
                "minor": "10"
              },
              "power_state": "Halted",
              "hasVendorDevice": false,
              "snapshots": [],
              "startDelay": 0,
              "startTime": null,
              "tags": [],
              "VIFs": [
                "0dde6812-79f8-327d-e69c-e098b6fec730"
              ],
              "virtualizationMode": "hvm",
              "$container": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
              "$VBDs": [
                "a2f64b99-6e5d-3cea-8331-fea0c91c0b8e",
                "f95fc772-a9fb-e944-8f7b-4aefc4d0124d"
              ],
              "VGPUs": [],
              "$VGPUs": [],
              "vga": "std",
              "videoram": "8",
              "coresPerSocket": 2,
              "snapshot_time": 1599054372,
              "id": "c830ce0b-d03d-9cb4-53ce-fbd0bc258208",
              "uuid": "c830ce0b-d03d-9cb4-53ce-fbd0bc258208",
              "$pool": "e78d04fc-b33f-6026-28bb-cdceb8e44863",
              "$poolId": "e78d04fc-b33f-6026-28bb-cdceb8e44863"
            }
            ]
            

            If there is anything else I can give you, please let me know..

            Kind Regards,
            ~Peg

            1 Reply Last reply Reply Quote 0
            • badrAZB Offline
              badrAZ
              last edited by

              Thank you for your response,

              The field snapshot_of doesn't exist. Please give me the output of

              xe vm-list uuid=snapshotUuid params=snapshot-of

              Thank you πŸ™‚

              AnonabharA 1 Reply Last reply Reply Quote 0
              • AnonabharA Offline
                Anonabhar @badrAZ
                last edited by

                Hi @badrAZ

                Thanks for your help. I got a really strange response (as in... Nothing...)

                cc7640ab-1ae4-4fd8-a395-a665b75f7bad-image.png

                [09:18 shineyg8-2-1 ~]# xe vm-list uuid=c830ce0b-d03d-9cb4-53ce-fbd0bc258208 params=snapshot-of
                [09:19 shineyg8-2-1 ~]#
                

                But if I do it against the original (base) VM I get:

                [09:17 shineyg8-2-1 ~]# xe vm-list uuid=547501f1-51e5-67ea-771d-304ad95145aa params=snapshot-of
                snapshot-of ( RO)    : <not in database>
                
                1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates πŸͺ Co-Founder CEO
                  last edited by

                  That's because you need to use xe snapshot-param-list.

                  AnonabharA 1 Reply Last reply Reply Quote 0
                  • AnonabharA Offline
                    Anonabhar @olivierlambert
                    last edited by

                    @olivierlambert & @badrAZ

                    Ah, that is better 8-) This is what I get this time...

                    [09:56 shineyg8-2-1 ~]# xe snapshot-param-list uuid=c830ce0b-d03d-9cb4-53ce-fbd0bc258208 
                    uuid ( RO)                                  : c830ce0b-d03d-9cb4-53ce-fbd0bc258208
                                                name-label ( RW): Test snapshot
                                          name-description ( RW): 
                                              user-version ( RW): 1
                                             is-a-template ( RW): true
                                       is-default-template ( RW): false
                                             is-a-snapshot ( RO): true
                                               snapshot-of ( RO): 547501f1-51e5-67ea-771d-304ad95145aa
                                                 snapshots ( RO): 
                                             snapshot-time ( RO): 20200902T13:46:12Z
                                             snapshot-info ( RO): disk-snapshot-type: crash_consistent; power-state-at-snapshot: Halted
                                                    parent ( RO): <not in database>
                                                  children ( RO): 547501f1-51e5-67ea-771d-304ad95145aa
                                         is-control-domain ( RO): false
                                               power-state ( RO): halted
                                             memory-actual ( RO): 0
                                             memory-target ( RO): 0
                                           memory-overhead ( RO): 20971520
                                         memory-static-max ( RW): 2147483648
                                        memory-dynamic-max ( RW): 2147483648
                                        memory-dynamic-min ( RW): 2147483648
                                         memory-static-min ( RW): 536870912
                                          suspend-VDI-uuid ( RW): <not in database>
                                           suspend-SR-uuid ( RW): <not in database>
                                              VCPUs-params (MRW): 
                                                 VCPUs-max ( RW): 2
                                          VCPUs-at-startup ( RW): 2
                                    actions-after-shutdown ( RW): Destroy
                                      actions-after-reboot ( RW): Restart
                                       actions-after-crash ( RW): Restart
                                             console-uuids (SRO): 
                                                       hvm ( RO): false
                                                  platform (MRW): timeoffset: 1; videoram: 8; hpet: true; secureboot: false; device-model: qemu-upstream-compat; apic: true; device_id: 0001; vga: std; nx: true; pae: true; viridian: false; acpi: 1; cores-per-socket: 2
                                        allowed-operations (SRO): destroy; export; revert; clone; copy
                                        current-operations (SRO): 
                                        blocked-operations (MRW): 
                                       allowed-VBD-devices (SRO): 1; 2; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254
                                       allowed-VIF-devices (SRO): 1; 2; 3; 4; 5; 6
                                            possible-hosts ( RO): ec347d65-341f-4ad2-941a-2e8544539ae7; 64e44a18-ca98-4014-9265-dc0854ffa6cd; 74d1a6d6-0660-4d4f-825d-60d609fc4fec; d4fb12f2-8d30-4efd-bf27-40006f6ec809; 92780235-d330-470f-88dd-b62d755ef119
                                               domain-type ( RW): hvm
                                       current-domain-type ( RO): unspecified
                                           HVM-boot-policy ( RW): BIOS order
                                           HVM-boot-params (MRW): firmware: bios; order: cdn
                                     HVM-shadow-multiplier ( RW): 1.000
                                                 PV-kernel ( RW): 
                                                PV-ramdisk ( RW): 
                                                   PV-args ( RW): 
                                            PV-legacy-args ( RW): 
                                             PV-bootloader ( RW): 
                                        PV-bootloader-args ( RW): 
                                       last-boot-CPU-flags ( RO): vendor: GenuineIntel; features: 1fcbfbff-97ba2223-2d93fbff-00000403-00000001-00000000-00000000-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000
                                          last-boot-record ( RO): ''
                                               resident-on ( RO): <not in database>
                                                  affinity ( RW): <not in database>
                                              other-config (MRW): import_task: OpaqueRef:66a61f13-3146-4c48-a99c-13ab36bf75c4; instant: true; base_template_name: Debian Buster 10; mac_seed: 40154d39-694a-3584-9863-dee48b5a5e68; install-methods: cdrom,nfs,http,ftp; linux_template: true
                                                    dom-id ( RO): -1
                                           recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="32"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions>
                                             xenstore-data (MRW): vm-data: ; vm-data/mmio-hole-size: 268435456
                                ha-always-run ( RW) [DEPRECATED]: false
                                       ha-restart-priority ( RW): 
                                                     blobs ( RO): 
                                                start-time ( RO): 19700101T00:00:00Z
                                              install-time ( RO): 19700101T00:00:00Z
                                              VCPUs-number ( RO): 0
                                         VCPUs-utilisation (MRO): 
                                                os-version (MRO): name: Debian GNU/Linux 10 (buster); uname: 4.19.0-9-amd64; distro: debian; major: 10; minor: 10
                                        PV-drivers-version (MRO): major: 8; minor: 0; micro: 50; build: 1
                        PV-drivers-up-to-date ( RO) [DEPRECATED]: true
                                                    memory (MRO): 
                                                     disks (MRO): 
                                                      VBDs (SRO): a2f64b99-6e5d-3cea-8331-fea0c91c0b8e; f95fc772-a9fb-e944-8f7b-4aefc4d0124d
                                                  networks (MRO): 0/ip: 185.192.178.68; 0/ipv4/0: 185.192.178.68; 0/ipv6/1: fe80::6c88:c3ff:fe40:be7e; 0/ipv6/0: 2a0a:2e40:ac01:802:6c88:c3ff:fe40:be7e
                                       PV-drivers-detected ( RO): true
                                                     other (MRO): platform-feature-multiprocessor-suspend: 1; has-vendor-device: 0; feature-suspend: 1; feature-poweroff: 1; feature-reboot: 1; feature-vcpu-hotplug: 1; feature-balloon: 1; shutdown: 
                                                      live ( RO): true
                                guest-metrics-last-updated ( RO): 20200823T20:32:37Z
                                       can-use-hotplug-vbd ( RO): unspecified
                                       can-use-hotplug-vif ( RO): unspecified
                                  cooperative ( RO) [DEPRECATED]: true
                                                      tags (SRW): 
                                                 appliance ( RW): <not in database>
                                         snapshot-schedule ( RW): <not in database>
                                          is-vmss-snapshot ( RO): false
                                               start-delay ( RW): 0
                                            shutdown-delay ( RW): 0
                                                     order ( RW): 0
                                                   version ( RO): 0
                                             generation-id ( RO): 
                                 hardware-platform-version ( RO): 0
                                         has-vendor-device ( RW): false
                                           requires-reboot ( RO): false
                                           reference-label ( RO): debian-10
                                              bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; baseboard-manufacturer: ; baseboard-product-name: ; baseboard-version: ; baseboard-serial-number: ; baseboard-asset-tag: ; baseboard-location-in-chassis: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d
                    
                    
                    [09:56 shineyg8-2-1 ~]#
                    
                    badrAZB 1 Reply Last reply Reply Quote 0
                    • badrAZB Offline
                      badrAZ @Anonabhar
                      last edited by badrAZ

                      @Anonabhar It seems that the property snapshot-of isn't correctly reported by XO. Did you tried to disconnect/reconnect the server containing this snapshot to see if it resolves your issue?

                      And please give me the version of your xo-server.

                      AnonabharA 1 Reply Last reply Reply Quote 1
                      • AnonabharA Offline
                        Anonabhar @badrAZ
                        last edited by

                        Hi @badrAZ

                        Well.. Monkey Poo!!!!! 😎

                        Disconnecting the server and re-enabling the server did clear the issue. I have made a few snapshots and run a few backups to cause snapshots to be rotated and all is looking good..

                        Something must have been stuck in the original connection?

                        FYI:
                        946986f1-a5ea-4680-a551-b0c953cde81f-image.png

                        Please mark this as solved...

                        ~Peg

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO Offline
                          olivierlambert Vates πŸͺ Co-Founder CEO
                          last edited by

                          Which isn't normal is the fact XO should "reset" the connection and fetch all objects again to avoid "partial" object info… @badrAZ we should investigate on that, we can't rely on just disconnect/reconnect in such situations.

                          1 Reply Last reply Reply Quote 0
                          • badrAZB Offline
                            badrAZ
                            last edited by

                            @olivierlambert I've created the dedicated Github issue πŸ‘

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post