XO spawned VM from template has wrong VIF network order
-
@olivierlambert Hey Oliver.
I have 2 networks configured in XO: Net1 and Net2.
Two different address pool and 2 different VLANs.
The order on the host supposed to be ETH0 -> Net1 and ETH1 -> Net2.
I have created VM with that network order and then converted it to template.
After spawning VM from that template the network order is Net2, Net1 which causes problem with my scripts.
Any reason/solution/ideas how to deal with it?Thanks
-
Please provide the config output of your template (
xe template-param-list uuid=<template UUID>) -
uuid ( RO) : 6f7611e7-00aa-07ac-66cc-4bf709576af7 name-label ( RW): Debian11-template name-description ( RW): [<env>/<DC>] <application/infra>: <app name/infra role> user-version ( RW): 1 is-a-template ( RW): true is-default-template ( RW): false is-a-snapshot ( RO): false snapshot-of ( RO): <not in database> snapshots ( RO): snapshot-time ( RO): 19700101T00:00:00Z snapshot-info ( RO): parent ( RO): <not in database> children ( RO): is-control-domain ( RO): false power-state ( RO): halted memory-actual ( RO): <not in database> memory-target ( RO): 0 memory-overhead ( RO): 19922944 memory-static-max ( RW): 2147483648 memory-dynamic-max ( RW): 2147483648 memory-dynamic-min ( RW): 2147483648 memory-static-min ( RW): 268435456 suspend-VDI-uuid ( RW): <not in database> suspend-SR-uuid ( RW): <not in database> VCPUs-params (MRW): VCPUs-max ( RW): 1 VCPUs-at-startup ( RW): 1 actions-after-shutdown ( RW): Destroy actions-after-reboot ( RW): Restart actions-after-crash ( RW): Restart console-uuids (SRO): hvm ( RO): false platform (MRW): timeoffset: 1; secureboot: false; device-model: qemu-upstream-compat; videoram: 8; hpet: true; apic: true; device_id: 0001; vga: std; nx: true; pae: true; viridian: false; acpi: 1 allowed-operations (SRO): changing_NVRAM; changing_dynamic_range; changing_shadow_memory; changing_static_range; migrate_send; provision; destroy; export; clone; copy current-operations (SRO): blocked-operations (MRW): allowed-VBD-devices (SRO): 1; 2; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254 allowed-VIF-devices (SRO): 2; 3; 4; 5; 6 possible-hosts ( RO): 6db7ad46-c045-41f8-b2b2-ab1d6ad845e3 domain-type ( RW): hvm current-domain-type ( RO): <not in database> HVM-boot-policy ( RW): BIOS order HVM-boot-params (MRW): firmware: bios; order: cdn HVM-shadow-multiplier ( RW): 1.000 PV-kernel ( RW): PV-ramdisk ( RW): PV-args ( RW): PV-legacy-args ( RW): PV-bootloader ( RW): PV-bootloader-args ( RW): last-boot-CPU-flags ( RO): vendor: GenuineIntel; features: 1fcbfbff-f7fa3223-2d93fbff-00000523-0000000f-019c47ab-00000008-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000 last-boot-record ( RO): '' resident-on ( RO): <not in database> affinity ( RW): <not in database> other-config (MRW): import_task: OpaqueRef:432bd179-55fa-4c25-bcbf-59a6245819c6; mac_seed: d506686a-830e-d4b4-3814-33868a1315ac; base_template_name: Debian Stretch 9.0; install-methods: cdrom,nfs,http,ftp; linux_template: true dom-id ( RO): -1 recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="32"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions> xenstore-data (MRW): vm-data/mmio-hole-size: 268435456; vm-data: ha-always-run ( RW) [DEPRECATED]: false ha-restart-priority ( RW): blobs ( RO): start-time ( RO): <unknown time> install-time ( RO): <unknown time> VCPUs-number ( RO): <not in database> VCPUs-utilisation (MRO): os-version (MRO): PV-drivers-version (MRO): PV-drivers-up-to-date ( RO) [DEPRECATED]: false memory (MRO): disks (MRO): VBDs (SRO): d5154254-a1dd-9873-2590-de8031f20614; 40f49a62-ad2c-1ecd-69f7-e39417f1b67c networks (MRO): PV-drivers-detected ( RO): false other (MRO): has-vendor-device: 0; platform-feature-xs_reset_watches: 1; platform-feature-multiprocessor-suspend: 1 live ( RO): true guest-metrics-last-updated ( RO): 20220622T13:42:25Z can-use-hotplug-vbd ( RO): unspecified can-use-hotplug-vif ( RO): unspecified cooperative ( RO) [DEPRECATED]: true tags (SRW): appliance ( RW): <not in database> snapshot-schedule ( RW): <not in database> is-vmss-snapshot ( RO): false start-delay ( RW): 0 shutdown-delay ( RW): 0 order ( RW): 0 version ( RO): 0 generation-id ( RO): hardware-platform-version ( RO): 0 has-vendor-device ( RW): false requires-reboot ( RO): false reference-label ( RO): debian-9 bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d -
I do not see any VIF order here.
Is there a way to force it on the template config level? -
@olivierlambert Anything else I may provide to help to fix this issue?
Thanks -
Each VIF got a device number (from 0 to n).
Do a
xe vm-vif-list uuid=<VM UUID>to see each VIF. -
@olivierlambert I can see template having allowed-VIF-devices list:
allowed-VIF-devices (SRO): 2; 3; 4; 5; 6but I do not see any list of preferred order of them.
Is there a way to force template to to use it in specific order?
And why would VM created based on the template have that order reversed? -
There's no reason the template will change the order. I'm still not sure to understand your problem at all
-
@olivierlambert I am surprised as well, that is why I have raised this question.
Template is saved with the networks order Net1 Net2.
VM spawned from it is having networks order Net2, Net1.Is there a way to check which order the template was created with?
Maybe the networks were renamed in the mean time or something?
I am shooting in the dark here ... -
On the created VM (from this template), do a
xe vm-vif-list uuid=<VM UUID>and paste it here. -
@olivierlambert
I have recreated the base VM [debian11-template], convert it to template and generated net VM [zzz-nettest2] based on that template.
These are the differences in VIF config:original VM:
uuid ( RO) : 84c4bd3e-4f50-48dc-4050-19dd4e8a8a8a vm-name-label ( RO): Debian11-template **device ( RO): 0** MAC ( RO): 9e:82:ec:4e:53:4f network-uuid ( RO): 1f7bec14-2ff5-471d-ba9c-bd84882ede17 **network-name-label ( RO): Net1** uuid ( RO) : 1d0aca18-e4f2-8b08-9ae6-022846241ab1 vm-name-label ( RO): Debian11-template **device ( RO): 1** MAC ( RO): 02:76:1f:22:c0:be network-uuid ( RO): 79fd2cb6-762a-e433-129a-edbb32612735 **network-name-label ( RO): Net2**new VM
uuid ( RO) : f4e200d6-2ba4-d2a4-961e-a0431951d25a vm-name-label ( RO): zzz-nettest2 **device ( RO): 1** MAC ( RO): 6e:18:1e:93:38:e9 network-uuid ( RO): 1f7bec14-2ff5-471d-ba9c-bd84882ede17 **network-name-label ( RO): Net1** uuid ( RO) : 16fbbd49-bb80-d111-dc83-5c47447ac174 vm-name-label ( RO): zzz-nettest2 **device ( RO): 0** MAC ( RO): a2:fe:cd:37:58:9f network-uuid ( RO): 79fd2cb6-762a-e433-129a-edbb32612735 **network-name-label ( RO): Net2**It happens no matter if I convert the template from the VM or create it from snapshot.
Template details below:
uuid ( RO) : f5b33b1c-798c-7f01-3ad6-9f5a44d1a25e name-label ( RW): Debian11-template name-description ( RW): [<env>/<DC>] <application/infra>: <app name/infra role> user-version ( RW): 1 is-a-template ( RW): true is-default-template ( RW): false is-a-snapshot ( RO): false snapshot-of ( RO): <not in database> snapshots ( RO): snapshot-time ( RO): 19700101T00:00:00Z snapshot-info ( RO): parent ( RO): <not in database> children ( RO): is-control-domain ( RO): false power-state ( RO): halted memory-actual ( RO): <not in database> memory-target ( RO): 0 memory-overhead ( RO): 19922944 memory-static-max ( RW): 2147483648 memory-dynamic-max ( RW): 2147483648 memory-dynamic-min ( RW): 2147483648 memory-static-min ( RW): 268435456 suspend-VDI-uuid ( RW): <not in database> suspend-SR-uuid ( RW): <not in database> VCPUs-params (MRW): VCPUs-max ( RW): 1 VCPUs-at-startup ( RW): 1 actions-after-shutdown ( RW): Destroy actions-after-reboot ( RW): Restart actions-after-crash ( RW): Restart console-uuids (SRO): hvm ( RO): false platform (MRW): timeoffset: 1; secureboot: false; device-model: qemu-upstream-compat; videoram: 8; hpet: true; apic: true; device_id: 0001; vga: std; nx: true; pae: true; viridian: false; acpi: 1 allowed-operations (SRO): changing_NVRAM; changing_dynamic_range; changing_shadow_memory; changing_static_range; migrate_send; provision; destroy; export; clone; copy current-operations (SRO): blocked-operations (MRW): allowed-VBD-devices (SRO): 1; 2; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254 allowed-VIF-devices (SRO): 2; 3; 4; 5; 6 possible-hosts ( RO): 9a33c146-c64d-4933-a54a-18941cb825f0 domain-type ( RW): hvm current-domain-type ( RO): <not in database> HVM-boot-policy ( RW): BIOS order HVM-boot-params (MRW): firmware: bios; order: cdn HVM-shadow-multiplier ( RW): 1.000 PV-kernel ( RW): PV-ramdisk ( RW): PV-args ( RW): PV-legacy-args ( RW): PV-bootloader ( RW): PV-bootloader-args ( RW): last-boot-CPU-flags ( RO): vendor: GenuineIntel; features: 1fcbfbff-f7fa3223-2d93fbff-00000523-0000000f-019c47ab-00000008-00000000-00001000-9c000400-00000000-00000000-00000000-00000000-00000000 last-boot-record ( RO): '' resident-on ( RO): <not in database> affinity ( RW): <not in database> other-config (MRW): import_task: OpaqueRef:eccf2505-7aaf-4592-b9b0-9887cfa77b33; mac_seed: 1c02ad94-8a62-f861-eabd-0baf9b7c32d4; base_template_name: Debian Stretch 9.0; install-methods: cdrom,nfs,http,ftp; linux_template: true dom-id ( RO): -1 recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="32"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions> xenstore-data (MRW): vm-data: ; vm-data/mmio-hole-size: 268435456 ha-always-run ( RW) [DEPRECATED]: false ha-restart-priority ( RW): blobs ( RO): start-time ( RO): <unknown time> install-time ( RO): <unknown time> VCPUs-number ( RO): <not in database> VCPUs-utilisation (MRO): os-version (MRO): PV-drivers-version (MRO): PV-drivers-up-to-date ( RO) [DEPRECATED]: false memory (MRO): disks (MRO): VBDs (SRO): fd0f2074-c6e8-b505-b5dc-95ac02fc4e42; 0bc0c5cd-afb5-54f3-8a48-815539029c4c networks (MRO): PV-drivers-detected ( RO): false other (MRO): platform-feature-multiprocessor-suspend: 1; has-vendor-device: 0 live ( RO): true guest-metrics-last-updated ( RO): 20220711T14:23:59Z can-use-hotplug-vbd ( RO): unspecified can-use-hotplug-vif ( RO): unspecified cooperative ( RO) [DEPRECATED]: true tags (SRW): appliance ( RW): <not in database> snapshot-schedule ( RW): <not in database> is-vmss-snapshot ( RO): false start-delay ( RW): 0 shutdown-delay ( RW): 0 order ( RW): 0 version ( RO): 0 generation-id ( RO): hardware-platform-version ( RO): 0 has-vendor-device ( RW): false requires-reboot ( RO): false reference-label ( RO): debian-9 bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/d43fa6d3fe8aaef04c2bad9d3e3daaef04c2bad9d3 -
@olivierlambert - any ideas?
Thanks
-
@olivierlambert - any ideas as I ran out of options.
-
Sorry I'm ultra busy at the moment and just getting out of COVID. Hopefully, someone else in the community can help you
-
@mietek Might need to change the NIC order? Should be possible in a similar way in XCP-ng as in XenServer; see for example: https://support.citrix.com/article/CTX135809/how-to-change-order-of-nics-in-xenserver
-
@olivierlambert
Thanks for the info and get well soon! -
@tjkreidl Thanks for the article. Unfortunately this machine is full of running VMs and changing NICs order on the host might have a global impact on all of them so it seems like an overkill.
I have created this template on this machine and created VM based on that template right away and I experience this issue. It looks like there is a problem with the template creation or template based VM creation process.
Is there a way to change it NIC order in the template itself? -
@mietek As to changing the NIC order in a template, not as far as I know. That is a host/pool configuration setting.
-
@tjkreidl This is surprising then that VM spawned from the template would even be able to change NIC order. This make no sense.
On the other hand how does the template know what is the list on NICs attached to the VM created from it and what should be the order of them then? -
@mietek Depends on the VIF that the VM had in the template, I would imagine. Easy enough to change once you have your new VM. Worst case, create a new template.
I have seen NICs randomly change order on hosts in the past - scary stuff!
