wttw

wttw

@bleader No, the opnsense box itself doesn't have wireguard (or anything else VPN-ish) running on it. It's mostly just a NAT with the normal variety of DHCP, DNS, ... services running on it.

wttw

I'm not clear on where it's most useful to report bugs in beta releases, so...

Yesterday I installed 8.3beta2 on bare hardware (NUC24OXGv9), installed orchestra and started migrating some VMs from ESXi.

While the second VM was being migrated, everything stopped.

At that point the two VMs that had been running - one vanilla ubuntu, and the orchestra VM - weren't any more, and it looked like dom0 had rebooted.

Looking at the crash logs the relevant snippets of dom0.log seem to be:

[   7955.734205]   INFO: block tdc: sector-size: 512/512 capacity: 125829120
[  10363.803886]   WARN: vif2.0: dropped over-mtu packet: 68785 > 1500
[  10363.803905]   WARN: WARNING: CPU: 0 PID: 8940 at lib/iov_iter.c:825 page_copy_sane.part.7+0x0/0x11
[  10363.803906]   WARN: Modules linked in: tun bnx2fc(O) cnic(O) uio fcoe libfcoe libfc scsi_transport_fc openvswitch nsh nf_nat_ipv6 nf_nat_ipv4 nf_conncount nf_nat 8021q garp mrp stp llc ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_multiport xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter dm_multipath sunrpc nls_iso8859_1 nls_cp437 intel_powerclamp crct10dif_pclmul crc32_pclmul ghash_clmulni_intel vfat pcbc fat aesni_intel aes_x86_64 crypto_simd dm_mod cryptd glue_helper video backlight ip_tables x_tables hid_generic usbhid hid xhci_pci igc(O) nvme xhci_hcd i40e(O) nvme_core scsi_dh_rdac scsi_dh_hp_sw scsi_dh_emc scsi_dh_alua scsi_mod efivarfs ipv6 crc_ccitt
[  10363.803925]   WARN: CPU: 0 PID: 8940 Comm: handler122 Tainted: G           O      4.19.0+1 #1
[  10363.803925]   WARN: Hardware name: Simply NUC NUC24OXGv9/AHWSA, BIOS AHWSA.1.23 04/12/2024
[  10363.803926]   WARN: RIP: e030:page_copy_sane.part.7+0x0/0x11

[...]

[  10363.806328]   WARN: CR2: 00007f0dd2f68000 CR3: 000000023e91e000 CR4: 0000000000040660
[  10363.806335]  EMERG: Kernel panic - not syncing: Fatal exception in interrupt

I've not yet been able to replicate the issue, and everything else seems to be working fine, including redoing the migration that was happening during the crash.

It looks similar to the bug referred to here - very scary host reboot issue.

The running linux VM was idle, but was connected to tailscale. Our edge device - the only thing other than the ESXi box and my macbook it would have been talking to - is running OPNSense.

wttw

@olivierlambert said in OVA import support?:

Yes it's supposed to work. Sadly, there's as much as different OVAs than VMs and VMware versions. Every time we are fixing new stuff when it's reported. Please provide the OVA somewhere so we can find why and fix it

https://tupid.org/debian11.ova - it's about 850MB.

I just created this on ESXi 7.0U3, then exported it with

./ovftool --noSSLVerify --diskMode=thin --targetType=ova --compress=9 "vi://skull/debian11" .

(VMware ovftool 4.6.2 (build-22220919))

Import from VMware should work if you have indeed the XCP-ng machine in extra for it.

The hardware is on order, but I thought I'd start testing this weekend.

But we could fix the OVA already, at least as soon we can check what's wrong with the XML.

Here's debian11.ovf, for anyone who wants to look without downloading the ova.

<?xml version="1.0" encoding="UTF-8"?>
<!--Generated by VMware ESX Server, User: root, UTC time: 2024-04-07T15:42:54.40766Z-->
<Envelope vmw:buildId="build-19482537" xmlns="http://schemas.dmtf.org/ovf/envelope/1" xmlns:cim="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <References>
    <File ovf:compression="gzip" ovf:href="debian11-disk1.vmdk.gz" ovf:id="file1" ovf:size="856235944"/>
    <File ovf:compression="gzip" ovf:href="debian11-file1.nvram.gz" ovf:id="file2" ovf:size="1821"/>
  </References>
  <DiskSection>
    <Info>Virtual disk information</Info>
    <Disk ovf:capacity="16" ovf:capacityAllocationUnits="byte * 2^30" ovf:diskId="vmdisk1" ovf:fileRef="file1" ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized" ovf:populatedSize="2324692992"/>
  </DiskSection>
  <NetworkSection>
    <Info>The list of logical networks</Info>
    <Network ovf:name="VM Network">
      <Description>The VM Network network</Description>
    </Network>
  </NetworkSection>
  <VirtualSystem ovf:id="debian11">
    <Info>A virtual machine</Info>
    <Name>debian11</Name>
    <OperatingSystemSection ovf:id="1" vmw:osType="debian11Guest">
      <Info>The kind of installed guest operating system</Info>
    </OperatingSystemSection>
    <VirtualHardwareSection>
      <Info>Virtual hardware requirements</Info>
      <System>
        <vssd:ElementName>Virtual Hardware Family</vssd:ElementName>
        <vssd:InstanceID>0</vssd:InstanceID>
        <vssd:VirtualSystemIdentifier>debian11</vssd:VirtualSystemIdentifier>
        <vssd:VirtualSystemType>vmx-19</vssd:VirtualSystemType>
      </System>
      <Item>
        <rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
        <rasd:Description>Number of Virtual CPUs</rasd:Description>
        <rasd:ElementName>1 virtual CPU(s)</rasd:ElementName>
        <rasd:InstanceID>1</rasd:InstanceID>
        <rasd:ResourceType>3</rasd:ResourceType>
        <rasd:VirtualQuantity>1</rasd:VirtualQuantity>
      </Item>
      <Item>
        <rasd:AllocationUnits>byte * 2^20</rasd:AllocationUnits>
        <rasd:Description>Memory Size</rasd:Description>
        <rasd:ElementName>2048MB of memory</rasd:ElementName>
        <rasd:InstanceID>2</rasd:InstanceID>
        <rasd:ResourceType>4</rasd:ResourceType>
        <rasd:VirtualQuantity>2048</rasd:VirtualQuantity>
      </Item>
      <Item>
        <rasd:Address>0</rasd:Address>
        <rasd:Description>SATA Controller</rasd:Description>
        <rasd:ElementName>SATA Controller 0</rasd:ElementName>
        <rasd:InstanceID>3</rasd:InstanceID>
        <rasd:ResourceSubType>vmware.sata.ahci</rasd:ResourceSubType>
        <rasd:ResourceType>20</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="slotInfo.pciSlotNumber" vmw:value="34"/>
      </Item>
      <Item>
        <rasd:Address>0</rasd:Address>
        <rasd:Description>SCSI Controller</rasd:Description>
        <rasd:ElementName>SCSI Controller 0</rasd:ElementName>
        <rasd:InstanceID>4</rasd:InstanceID>
        <rasd:ResourceSubType>VirtualSCSI</rasd:ResourceSubType>
        <rasd:ResourceType>6</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="slotInfo.pciSlotNumber" vmw:value="160"/>
      </Item>
      <Item ovf:required="false">
        <rasd:Address>0</rasd:Address>
        <rasd:Description>USB Controller (EHCI)</rasd:Description>
        <rasd:ElementName>USB Controller</rasd:ElementName>
        <rasd:InstanceID>5</rasd:InstanceID>
        <rasd:ResourceSubType>vmware.usb.ehci</rasd:ResourceSubType>
        <rasd:ResourceType>23</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="slotInfo.pciSlotNumber" vmw:value="32"/>
        <vmw:Config ovf:required="false" vmw:key="slotInfo.ehciPciSlotNumber" vmw:value="33"/>
        <vmw:Config ovf:required="false" vmw:key="autoConnectDevices" vmw:value="false"/>
        <vmw:Config ovf:required="false" vmw:key="ehciEnabled" vmw:value="true"/>
      </Item>
      <Item>
        <rasd:Address>1</rasd:Address>
        <rasd:Description>IDE Controller</rasd:Description>
        <rasd:ElementName>VirtualIDEController 1</rasd:ElementName>
        <rasd:InstanceID>6</rasd:InstanceID>
        <rasd:ResourceType>5</rasd:ResourceType>
      </Item>
      <Item>
        <rasd:Address>0</rasd:Address>
        <rasd:Description>IDE Controller</rasd:Description>
        <rasd:ElementName>VirtualIDEController 0</rasd:ElementName>
        <rasd:InstanceID>7</rasd:InstanceID>
        <rasd:ResourceType>5</rasd:ResourceType>
      </Item>
      <Item ovf:required="false">
        <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
        <rasd:ElementName>VirtualVideoCard</rasd:ElementName>
        <rasd:InstanceID>8</rasd:InstanceID>
        <rasd:ResourceType>24</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="useAutoDetect" vmw:value="true"/>
        <vmw:Config ovf:required="false" vmw:key="videoRamSizeInKB" vmw:value="4096"/>
        <vmw:Config ovf:required="false" vmw:key="enable3DSupport" vmw:value="false"/>
        <vmw:Config ovf:required="false" vmw:key="use3dRenderer" vmw:value="automatic"/>
        <vmw:Config ovf:required="false" vmw:key="graphicsMemorySizeInKB" vmw:value="262144"/>
      </Item>
      <Item ovf:required="false">
        <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
        <rasd:ElementName>VirtualVMCIDevice</rasd:ElementName>
        <rasd:InstanceID>9</rasd:InstanceID>
        <rasd:ResourceSubType>vmware.vmci</rasd:ResourceSubType>
        <rasd:ResourceType>1</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="allowUnrestrictedCommunication" vmw:value="false"/>
      </Item>
      <Item ovf:required="false">
        <rasd:AddressOnParent>0</rasd:AddressOnParent>
        <rasd:AutomaticAllocation>false</rasd:AutomaticAllocation>
        <rasd:ElementName>CD-ROM 1</rasd:ElementName>
        <rasd:InstanceID>10</rasd:InstanceID>
        <rasd:Parent>3</rasd:Parent>
        <rasd:ResourceSubType>vmware.cdrom.atapi</rasd:ResourceSubType>
        <rasd:ResourceType>15</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="connectable.allowGuestControl" vmw:value="true"/>
      </Item>
      <Item>
        <rasd:AddressOnParent>0</rasd:AddressOnParent>
        <rasd:ElementName>Hard Disk 1</rasd:ElementName>
        <rasd:HostResource>ovf:/disk/vmdisk1</rasd:HostResource>
        <rasd:InstanceID>11</rasd:InstanceID>
        <rasd:Parent>4</rasd:Parent>
        <rasd:ResourceType>17</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="backing.writeThrough" vmw:value="false"/>
      </Item>
      <Item>
        <rasd:AddressOnParent>7</rasd:AddressOnParent>
        <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
        <rasd:Connection>VM Network</rasd:Connection>
        <rasd:Description>VmxNet3 ethernet adapter on &quot;VM Network&quot;</rasd:Description>
        <rasd:ElementName>Ethernet 1</rasd:ElementName>
        <rasd:InstanceID>12</rasd:InstanceID>
        <rasd:ResourceSubType>VmxNet3</rasd:ResourceSubType>
        <rasd:ResourceType>10</rasd:ResourceType>
        <vmw:Config ovf:required="false" vmw:key="slotInfo.pciSlotNumber" vmw:value="192"/>
        <vmw:Config ovf:required="false" vmw:key="wakeOnLanEnabled" vmw:value="false"/>
        <vmw:Config ovf:required="false" vmw:key="connectable.allowGuestControl" vmw:value="true"/>
      </Item>
      <vmw:Config ovf:required="false" vmw:key="cpuHotAddEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="cpuHotRemoveEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="memoryHotAddEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="firmware" vmw:value="bios"/>
      <vmw:Config ovf:required="false" vmw:key="cpuAllocation.shares.shares" vmw:value="1000"/>
      <vmw:Config ovf:required="false" vmw:key="cpuAllocation.shares.level" vmw:value="normal"/>
      <vmw:Config ovf:required="false" vmw:key="tools.syncTimeWithHost" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="tools.syncTimeWithHostAllowed" vmw:value="true"/>
      <vmw:Config ovf:required="false" vmw:key="tools.afterPowerOn" vmw:value="true"/>
      <vmw:Config ovf:required="false" vmw:key="tools.afterResume" vmw:value="true"/>
      <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestShutdown" vmw:value="true"/>
      <vmw:Config ovf:required="false" vmw:key="tools.beforeGuestStandby" vmw:value="true"/>
      <vmw:Config ovf:required="false" vmw:key="tools.toolsUpgradePolicy" vmw:value="manual"/>
      <vmw:Config ovf:required="false" vmw:key="powerOpInfo.powerOffType" vmw:value="soft"/>
      <vmw:Config ovf:required="false" vmw:key="powerOpInfo.resetType" vmw:value="soft"/>
      <vmw:Config ovf:required="false" vmw:key="powerOpInfo.suspendType" vmw:value="soft"/>
      <vmw:Config ovf:required="false" vmw:key="nestedHVEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="vPMCEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="virtualICH7MPresent" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="virtualSMCPresent" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="flags.vvtdEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="flags.vbsEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="bootOptions.efiSecureBootEnabled" vmw:value="false"/>
      <vmw:Config ovf:required="false" vmw:key="powerOpInfo.standbyAction" vmw:value="checkpoint"/>
      <vmw:ExtraConfig ovf:required="false" vmw:key="nvram" vmw:value="ovf:/file/file2"/>
      <vmw:ExtraConfig ovf:required="false" vmw:key="svga.autodetect" vmw:value="TRUE"/>
    </VirtualHardwareSection>
  </VirtualSystem>
</Envelope>

wttw

I have some OVA packages exported from a relatively recent ESXi setup.

I'm trying to import them into xcp-ng, using either the xen orchestra web interface or the upload-ova script. I've tried several approaches with very limited success.

If I use the web interface (either installed from the quick deploy, or a build from github HEAD running locally on my mac) I can drag the ova file onto the upload area, get all the metadata displayed, but when I click the "Import" button it does nothing. Checking the browser console logs I see it's silently failed with:

TypeError: undefined is not an object (evaluating 'forwardsInflater.result.length')
resultStart — ova-read.js:306
(anonymous function) — ova-read.js:315

If I use the upload-ova tool instead, I get more interesting errors.

data {
  tables: { 'bastion-disk1.vmdk.gz': Promise { <pending> } },
  disks: {
    vmdisk1: {
      capacity: 171798691840,
      path: 'bastion-disk1.vmdk.gz',
      compression: 'gzip',
      descriptionLabel: 'No description',
      nameLabel: 'Hard Disk 1',
      position: 0
    }
  },
  networks: [
    '8283276e-2380-72cc-9920-72f755637053',
    '8283276e-2380-72cc-9920-72f755637053'
  ],
  nameLabel: 'bastion',
  descriptionLabel: undefined,
  nCpus: 1,
  memory: 8589934592
}
✖ invalid parameters
  property @: must have required property 'sr'
  property @./data: must have required property 'descriptionLabel'
  property @./data/disks/0: must NOT have additional properties

and in the orchestrator logs I see more details:

vm.import
{
  "data": {
    "tables": {
      "bastion-disk1.vmdk.gz": {}
    },
    "disks": [
      {
        "capacity": 171798691840,
        "path": "bastion-disk1.vmdk.gz",
        "compression": "gzip",
        "descriptionLabel": "No description",
        "nameLabel": "Hard Disk 1",
        "position": 0
      }
    ],
    "networks": [
      "8283276e-2380-72cc-9920-72f755637053",
      "8283276e-2380-72cc-9920-72f755637053"
    ],
    "nameLabel": "bastion",
    "nCpus": 1,
    "memory": 8589934592
  },
  "type": "ova"
}
{
  "code": 10,
  "data": {
    "errors": [
      {
        "instancePath": "",
        "schemaPath": "#/required",
        "keyword": "required",
        "params": {
          "missingProperty": "sr"
        },
        "message": "must have required property 'sr'"
      },
      {
        "instancePath": "/data",
        "schemaPath": "#/properties/data/required",
        "keyword": "required",
        "params": {
          "missingProperty": "descriptionLabel"
        },
        "message": "must have required property 'descriptionLabel'"
      },
      {
        "instancePath": "/data/disks/0",
        "schemaPath": "#/properties/data/properties/disks/items/additionalProperties",
        "keyword": "additionalProperties",
        "params": {
          "additionalProperty": "compression"
        },
        "message": "must NOT have additional properties"
      }
    ]
  },
  "message": "invalid parameters",
  "name": "XoError",
  "stack": "XoError: invalid parameters
    at Module.invalidParameters (/Users/steve/3rd/xen-orchestra/packages/xo-common/api-errors.js:26:11)
    at Xo.call (file:///Users/steve/3rd/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:90:22)
    at Api.#callApiMethod (file:///Users/steve/3rd/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:362:19)"
}

If I try and use --override to set one of the missing parameters I get:

➜ ./index.js --upload /Volumes/local/backups/esxi/bastion.ova --override descriptionLabel=whatever
✖ Cannot read properties of undefined (reading '$poolId')
TypeError: Cannot read properties of undefined (reading '$poolId')
    at Object.upload (/Users/steve/3rd/xen-orchestra/@xen-orchestra/upload-ova/dist/index.js:168:12)

If I import the OVA into virtualbox and then export it again then I can import it via the orchestra web upload with no issues, so it's clearly a problem with the structure of OVAs produced by VMWare.

On to the questions:

Is this supposed to work? i.e. is this a set of bugs or am I trying to do something that's not supported?

Should I be doing something different when creating the OVAs, either with different settings in ovftool or by editing the generated files?

Is this path for importing from VMWare something I should be using at all, or should I be using the "Import from VMWare" option to suck the brains out of a running vSphere instance? (I've not tried this, as I'm trying to migrate from an ESXi box to an xcp-ng box and don't have the hardware available to run both at the moment ¯_(ツ)_/¯).

wttw

@wttw

Latest posts made by wttw