[SOLVED] Upgraded Arch VM kernel -- now xcp-ng will not boot VM

kevdog

I had Arch VM with ZFS on root running within xcp-VM. Recently upgraded kernel and now VM will not boot. (Not sure if this is kernel related however since a few other packages were upgraded as well).

When trying to start the VM, I get the following errors from the xcp-ng (I actually pulled these from Xen orchestra logs):

vm.start
{
  "id": "98a8dc5a-66f4-a475-ec2a-3de2df439629"
}
{
  "code": "VDI_MISSING",
  "params": [
    "OpaqueRef:3d2fb686-ebd9-4e68-b93b-c036bd2f1c99",
    "OpaqueRef:07784b90-20b0-4960-8e0b-4c4dc575db55"
  ],
  "call": {
    "method": "VM.start",
    "params": [
      "OpaqueRef:d091d6c6-168a-4f6a-b365-c54cac642031",
      false,
      false
    ]
  },
  "message": "VDI_MISSING(OpaqueRef:3d2fb686-ebd9-4e68-b93b-c036bd2f1c99, OpaqueRef:07784b90-20b0-4960-8e0b-4c4dc575db55)",
  "name": "XapiError",
  "stack": "XapiError: VDI_MISSING(OpaqueRef:3d2fb686-ebd9-4e68-b93b-c036bd2f1c99, OpaqueRef:07784b90-20b0-4960-8e0b-4c4dc575db55)
    at Function.wrap (/opt/xen-orchestra/packages/xen-api/src/_XapiError.js:16:11)
    at _httpRequestPlus.default.post.readAll.then.text (/opt/xen-orchestra/packages/xen-api/src/transports/json-rpc.js:35:26)
    at tryCatcher (/opt/xen-orchestra/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:15:14)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)"
}

With error complaining about VDI missing (which is weird), here is my VM-list:

xe vm-list
uuid ( RO)           : d7d5df9e-7ef0-1884-ba83-afdcf0c432ab
     name-label ( RW): [XO Backup pfSense/XO Backup] Ubuntu Xen Orchestra
    power-state ( RO): halted


uuid ( RO)           : 834f3e6f-a4c1-dfd5-c785-673c00fef3c6
     name-label ( RW): [XO Backup pfSense/XO Backup] pfSense
    power-state ( RO): halted


uuid ( RO)           : 98a8dc5a-66f4-a475-ec2a-3de2df439629
     name-label ( RW): Arch Time Machine
    power-state ( RO): halted


uuid ( RO)           : 3bdafe58-3c3d-daa6-55c5-0151aa17e0ad
     name-label ( RW): Ubuntu Xen Orchestra
    power-state ( RO): running


uuid ( RO)           : f9753f68-3d0e-862a-3912-20f7d5f72e62
     name-label ( RW): pfSense
    power-state ( RO): running


uuid ( RO)           : c2cd974f-c490-8860-6672-df6c57e18421
     name-label ( RW): pfSense_2019-09-28T15:23:41.310Z
    power-state ( RO): halted


uuid ( RO)           : fe7c11c6-07c5-2bb5-d31a-bebc42e41f33
     name-label ( RW): Ubuntu Test
    power-state ( RO): halted


uuid ( RO)           : 4b44b9ea-3da5-7a12-9fbf-d1b150cc8d85
     name-label ( RW): Ubuntu Test
    power-state ( RO): running


uuid ( RO)           : eebbe30a-833d-2268-f8e1-fee1f716a70b
     name-label ( RW): [XO Backup Arch-TM] Arch Time Machine
    power-state ( RO): halted


uuid ( RO)           : 43cbe5ad-0bed-0c41-c301-d924a3b5c809
     name-label ( RW): Ubuntu Test
    power-state ( RO): halted


uuid ( RO)           : 0aa694ad-7db9-5b5d-4047-14b532a0a2fd
     name-label ( RW): [XO Backup XO Delta Backup] Ubuntu Xen Orchestra
    power-state ( RO): halted


uuid ( RO)           : 821490d8-5858-f3d1-4c09-b2271aa85da5
     name-label ( RW): Ubuntu Test
    power-state ( RO): halted


uuid ( RO)           : 8d916a65-47b3-4424-917f-8ac9cf474b3c
     name-label ( RW): Control domain on host: xcp-ng-gohilton
    power-state ( RO): running


uuid ( RO)           : 138f595f-32eb-3bb9-95d6-1adb285c4b6b
     name-label ( RW): [XO Backup NonEssential VM] Ubuntu Test
    power-state ( RO): halted


uuid ( RO)           : 4d480db7-f21c-83d5-7db7-b9a8f07775ea
     name-label ( RW): Ubuntu Test
    power-state ( RO): halted

I tried to boot the VM manually by ID:

#xe vm-start uuid=98a8dc5a-66f4-a475-ec2a-3de2df439629
This operation cannot be performed because the specified VDI could not be found on the storage substrate
sr: 28a8b360-7b1b-58ca-81b6-f907c164ec38 (XCP-ng Tools)
vdi: b1a7a49a-f17c-42ac-af73-4429eb57b07a (Old version of guest-tools.iso)

The only other Arch VM listed is a snapshot:

# xe vm-start uuid=eebbe30a-833d-2268-f8e1-fee1f716a70b
Error code: VM_IS_SNAPSHOT
Error parameters: eebbe30a-833d-2268-f8e1-fee1f716a70b ([XO Backup Arch-TM] Arch Time Machine), start

kevdog

I guess I'll label this one solved. Turns out I needed to eject the virtual iso within the virtual CD-ROM drive. Perhaps VM boot order tries to boot from DVD First (although hard drive is listed first in the boot order of XO followed by DVD followed by Network.

Not sure but I'll label this one as solved.

olivierlambert

The message was relatively clear (for a XAPI error

This operation cannot be performed because the specified VDI could not be found on the storage substrate
sr: 28a8b360-7b1b-58ca-81b6-f907c164ec38 (XCP-ng Tools)
vdi: b1a7a49a-f17c-42ac-af73-4429eb57b07a (Old version of guest-tools.iso)

This means the VM can't boot with this current CD because it's not there anymore. I suppose you updated your host without ejecting all CDs before. So the old tools CD was removed for the new one, and your VM kept it. So when you tried to boot, it couldn't because it couldn't find the old ISO.