XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Import from VMware fails after upgrade to XOA 5.91

    Scheduled Pinned Locked Moved Xen Orchestra
    64 Posts 8 Posters 14.2k Views 9 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • J Offline
      jasonmap
      last edited by

      I'm running the default XOA image that was auto-deployed from my XCP-ng node on 'latest' (obv) - so it's debian.

      Here's the error block from journalctl:

      Feb 01 11:50:18 xoa xo-server[107155]: 2024-02-01T16:50:18.569Z xo:xo-server WARN possibly unhandled rejection {
Feb 01 11:50:18 xoa xo-server[107155]:   error: Error: already finalized or destroyed
Feb 01 11:50:18 xoa xo-server[107155]:       at Pack.entry (/usr/local/lib/node_modules/xo-server/node_modules/tar-stream/pack.js:138:51)
Feb 01 11:50:18 xoa xo-server[107155]:       at Pack.resolver (/usr/local/lib/node_modules/xo-server/node_modules/promise-toolbox/fromCallback.js:5:6)
Feb 01 11:50:18 xoa xo-server[107155]:       at Promise._execute (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/debuggability.js:384:9)
Feb 01 11:50:18 xoa xo-server[107155]:       at Promise._resolveFromExecutor (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:518:18)
Feb 01 11:50:18 xoa xo-server[107155]:       at new Promise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:103:10)
Feb 01 11:50:18 xoa xo-server[107155]:       at Pack.fromCallback (/usr/local/lib/node_modules/xo-server/node_modules/promise-toolbox/fromCallback.js:9:10)
Feb 01 11:50:18 xoa xo-server[107155]:       at writeBlock (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/xva/_writeDisk.mjs:11:22)
Feb 01 11:50:18 xoa xo-server[107155]: }
Feb 01 11:50:20 xoa xo-server[107155]: root@10.96.22.111 Xapi#putResource /import/ XapiError: IMPORT_ERROR(INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ])
Feb 01 11:50:20 xoa xo-server[107155]:     at Function.wrap (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_XapiError.mjs:16:12)
Feb 01 11:50:20 xoa xo-server[107155]:     at default (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_getTaskResult.mjs:11:29)
Feb 01 11:50:20 xoa xo-server[107155]:     at Xapi._addRecordToCache (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1006:24)
Feb 01 11:50:20 xoa xo-server[107155]:     at file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1040:14
Feb 01 11:50:20 xoa xo-server[107155]:     at Array.forEach (<anonymous>)
Feb 01 11:50:20 xoa xo-server[107155]:     at Xapi._processEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1030:12)
Feb 01 11:50:20 xoa xo-server[107155]:     at Xapi._watchEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1203:14) {
Feb 01 11:50:20 xoa xo-server[107155]:   code: 'IMPORT_ERROR',
Feb 01 11:50:20 xoa xo-server[107155]:   params: [ 'INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ]' ],
Feb 01 11:50:20 xoa xo-server[107155]:   call: undefined,
Feb 01 11:50:20 xoa xo-server[107155]:   url: undefined,
Feb 01 11:50:20 xoa xo-server[107155]:   task: task {
Feb 01 11:50:20 xoa xo-server[107155]:     uuid: 'd4267c54-3480-a7fe-4d6d-6cc18eef1dc9',
Feb 01 11:50:20 xoa xo-server[107155]:     name_label: '[XO] VM import',
Feb 01 11:50:20 xoa xo-server[107155]:     name_description: '',
Feb 01 11:50:20 xoa xo-server[107155]:     allowed_operations: [],
Feb 01 11:50:20 xoa xo-server[107155]:     current_operations: {},
Feb 01 11:50:20 xoa xo-server[107155]:     created: '20240201T16:44:17Z',
Feb 01 11:50:20 xoa xo-server[107155]:     finished: '20240201T16:50:23Z',
Feb 01 11:50:20 xoa xo-server[107155]:     status: 'failure',
Feb 01 11:50:20 xoa xo-server[107155]:     resident_on: 'OpaqueRef:85a049dc-296e-4ef0-bdbc-82e2845ecd68',
Feb 01 11:50:20 xoa xo-server[107155]:     progress: 1,
Feb 01 11:50:20 xoa xo-server[107155]:     type: '<none/>',
Feb 01 11:50:20 xoa xo-server[107155]:     result: '',
Feb 01 11:50:20 xoa xo-server[107155]:     error_info: [
Feb 01 11:50:20 xoa xo-server[107155]:       'IMPORT_ERROR',
Feb 01 11:50:20 xoa xo-server[107155]:       'INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ]'
Feb 01 11:50:20 xoa xo-server[107155]:     ],
Feb 01 11:50:20 xoa xo-server[107155]:     other_config: { object_creation: 'complete' },
Feb 01 11:50:20 xoa xo-server[107155]:     subtask_of: 'OpaqueRef:NULL',
Feb 01 11:50:20 xoa xo-server[107155]:     subtasks: [],
Feb 01 11:50:20 xoa xo-server[107155]:     backtrace: '(((process xapi)(filename lib/backtrace.ml)(line 210))((process xapi)(filename ocaml/xapi/import.ml)(line 2021))((proces>
Feb 01 11:50:20 xoa xo-server[107155]:   }
Feb 01 11:50:20 xoa xo-server[107155]: }
Feb 01 11:50:20 xoa xo-server[107155]: 2024-02-01T16:50:20.571Z xo:api WARN admin | vm.importMultipleFromEsxi(...) [6m] =!> Error: no opaque ref found

      Was the VM created ? Can I prepare a specific branch ( with more debug) for you to test ? You'll need to do a git checkout <branch_name> , and then restart xo-server ?

      No? The VM doesn't show up in XOA if that's what you're asking.

      I might need a little hand-holding on doing this part as I'm pretty new to the platform.

      1 Reply Last reply Reply Quote 1
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        @jasonmap said in Import from VMware fails after upgrade to XOA 5.91:

        'INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ]'

        This means you likely have a disk space problem on your Dom0. Can you do a df -h in your XCP-ng host?

        J 1 Reply Last reply Reply Quote 0
        • J Offline
          jasonmap @olivierlambert
          last edited by

          This means you likely have a disk space problem on your Dom0. Can you do a df -h in your XCP-ng host?

          Sure, it seems ok to me:

          [12:21 roentgenium ~]# df -h
Filesystem                                                            Size  Used Avail Use% Mounted on
devtmpfs                                                              2.0G   28K  2.0G   1% /dev
tmpfs                                                                 2.1G  344K  2.1G   1% /dev/shm
tmpfs                                                                 2.1G  9.4M  2.0G   1% /run
tmpfs                                                                 2.1G     0  2.1G   0% /sys/fs/cgroup
/dev/sda1                                                              18G  2.9G   14G  18% /
xenstore                                                              2.1G     0  2.1G   0% /var/lib/xenstored
/dev/sda5                                                             3.9G  302M  3.4G   9% /var/log
10.96.25.20:/mnt/seaborgium/XCP/41b662ff-93bd-c386-9dd0-5d08daa3f4e3   39T  1.0M   39T   1% /run/sr-mount/41b662ff-93bd-c386-9dd0-5d08daa3f4e3
10.96.22.19:/mnt/array1/Isos                                           29T   14T   16T  47% /run/sr-mount/f60530ef-04e2-daf4-36df-c24527814e97
10.96.35.31:/mnt/strontium/XCP/50d8f945-8ae4-dd87-0149-e6054a10d51f    52T   53G   52T   1% /run/sr-mount/50d8f945-8ae4-dd87-0149-e6054a10d51f
tmpfs                                                                 412M     0  412M   0% /run/user/0
          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            And in your XOA?

            J 1 Reply Last reply Reply Quote 0
            • J Offline
              jasonmap @olivierlambert
              last edited by

              @olivierlambert

              [12:25 01] xoa:log$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            954M     0  954M   0% /dev
tmpfs           195M  528K  195M   1% /run
/dev/xvda1       19G  3.1G   15G  18% /
tmpfs           973M     0  973M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           195M     0  195M   0% /run/user/1000
              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                Weird, it's like the tar stream with ended because of not enough space 🤔 But it doesn't make sense reading your available space.

                1 Reply Last reply Reply Quote 0
                • R Offline
                  rmaclachlan
                  last edited by

                  I started experiencing this exact same error after upgrading to 5.91 yesterday. It seems to affect one of my VMs that has 2 VMDK disks, the first (OS) disk copies over successfully, the 2nd (data) disk fails.

                  I've tried making a new VMDK and copying my data over, then destroying the old one in case there was a problem with that file but it still hangs around 50% then the task fails.

                  1 Reply Last reply Reply Quote 1
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    Might worth some investigation on our side, yes.

                    J 1 Reply Last reply Reply Quote 0
                    • J Offline
                      jasonmap @olivierlambert
                      last edited by

                      My VM doesn't have multiple vmdks, just the primary 16GB thin provisioned OS drive. Its actual size is only 4.77GB.

                      1vCPU
                      2GB RAM
                      16GB Disk
                      1 NIC
                      No dangling snapshots
                      powered off

                      truly nothing out of the ordinary.

                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        Yeah it seems simple, I wonder what's going on, @florent will take a look. Ideally, if you can open a support ticket so we can take a look remotely, that would be helpful 🙂

                        1 Reply Last reply Reply Quote 0
                        • florentF Offline
                          florent Vates 🪐 XO Team
                          last edited by

                          ok, I am testing it with more disks , but the ENOSPC puzzle me for now

                          1 Reply Last reply Reply Quote 0
                          • florentF Offline
                            florent Vates 🪐 XO Team
                            last edited by

                            @jasonmap , @rmaclachlan , can you test this branch : fix_xva_import_thin ?
                            from this PR : https://github.com/vatesfr/xen-orchestra/pull/7354

                            fbeauchamp opened this pull request in vatesfr/xen-orchestra

                            closed fix(xo-server/migrate-vms): XVA import - fixes omnibus #7354

                            J 1 Reply Last reply Reply Quote 0
                            • J Offline
                              jasonmap @florent
                              last edited by

                              @florent

                              I would be happy to test - but this is where that "hand holding" would come into play.

                              florentF 1 Reply Last reply Reply Quote 0
                              • florentF Offline
                                florent Vates 🪐 XO Team @jasonmap
                                last edited by julien-f

                                @jasonmap said in Import from VMware fails after upgrade to XOA 5.91:

                                @florent

                                I would be happy to test - but this is where that "hand holding" would come into play.

                                no problem . If you have a XOA , please open a support tunnel, I can patch your install
                                If not:

                                1. go to your XO install folder, inside, you should have a lot of files, and also a @vates, a @xen-orchestra and a packages folder
                                2. git checkout fix_xva_import_thin
                                3. yarn && yarn build
                                  4 restart your xo-server
                                A J 2 Replies Last reply Reply Quote 0
                                • A Offline
                                  archw @florent
                                  last edited by julien-f

                                  @florent
                                  In the FWIW department, I get the same errors when trying to import with 5.91:

                                  result	
succeeded	{}
message	"no opaque ref found"
name	"Error"
stack	"Error: no opaque ref found\n    at importVm (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/xva/importVm.mjs:28:19)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at importVdi (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/xva/importVdi.mjs:6:17)\n    at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/migrate-vm.mjs:260:21\n    at Task.runInside (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:158:22)\n    at Task.run (/usr/local/lib/node_modules/xo-server/node_modules/@vates/task/index.js:141:20)"

                                  The journalctl errors are as follows :

                                  Feb 02 09:30:19 xoa xo-server[1178375]: 2024-02-02T14:30:19.498Z xo:xo-server WARN possibly unhandled rejection {
Feb 02 09:30:19 xoa xo-server[1178375]:   error: Error: already finalized or destroyed
Feb 02 09:30:19 xoa xo-server[1178375]:       at Pack.entry (/usr/local/lib/node_modules/xo-server/node_modules/tar-stream/pack.js:138:51)
Feb 02 09:30:19 xoa xo-server[1178375]:       at Pack.resolver (/usr/local/lib/node_modules/xo-server/node_modules/promise-toolbox/fromCallback.js:5:6)
Feb 02 09:30:19 xoa xo-server[1178375]:       at Promise._execute (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/debuggability.js:384:9)
Feb 02 09:30:19 xoa xo-server[1178375]:       at Promise._resolveFromExecutor (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:518:18)
Feb 02 09:30:19 xoa xo-server[1178375]:       at new Promise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:103:10)
Feb 02 09:30:19 xoa xo-server[1178375]:       at Pack.fromCallback (/usr/local/lib/node_modules/xo-server/node_modules/promise-toolbox/fromCallback.js:9:10)
Feb 02 09:30:19 xoa xo-server[1178375]:       at writeBlock (file:///usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/xva/_writeDisk.mjs:11:22)
Feb 02 09:30:19 xoa xo-server[1178375]: }
Feb 02 09:30:21 xoa xo-server[1178375]: root@xx.xx.xx.xx Xapi#putResource /import/ XapiError: IMPORT_ERROR(INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ])
Feb 02 09:30:21 xoa xo-server[1178375]:     at Function.wrap (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_XapiError.mjs:16:12)
Feb 02 09:30:21 xoa xo-server[1178375]:     at default (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_getTaskResult.mjs:11:29)
Feb 02 09:30:21 xoa xo-server[1178375]:     at Xapi._addRecordToCache (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1006:24)
Feb 02 09:30:21 xoa xo-server[1178375]:     at file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1040:14
Feb 02 09:30:21 xoa xo-server[1178375]:     at Array.forEach (<anonymous>)
Feb 02 09:30:21 xoa xo-server[1178375]:     at Xapi._processEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1030:12)
Feb 02 09:30:21 xoa xo-server[1178375]:     at Xapi._watchEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1203:14)
Feb 02 09:30:21 xoa xo-server[1178375]:     at runNextTicks (node:internal/process/task_queues:60:5)
Feb 02 09:30:21 xoa xo-server[1178375]:     at processImmediate (node:internal/timers:447:9)
Feb 02 09:30:21 xoa xo-server[1178375]:     at process.callbackTrampoline (node:internal/async_hooks:130:17) {
Feb 02 09:30:21 xoa xo-server[1178375]:   code: 'IMPORT_ERROR',
Feb 02 09:30:21 xoa xo-server[1178375]:   params: [ 'INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ]' ],
Feb 02 09:30:21 xoa xo-server[1178375]:   call: undefined,
Feb 02 09:30:21 xoa xo-server[1178375]:   url: undefined,
Feb 02 09:30:21 xoa xo-server[1178375]:   task: task {
Feb 02 09:30:21 xoa xo-server[1178375]:     uuid: '37b70fee-d769-1b2a-864b-7e18bd6fcff6',
Feb 02 09:30:21 xoa xo-server[1178375]:     name_label: '[XO] VM import',
Feb 02 09:30:21 xoa xo-server[1178375]:     name_description: '',
Feb 02 09:30:21 xoa xo-server[1178375]:     allowed_operations: [],
Feb 02 09:30:21 xoa xo-server[1178375]:     current_operations: {},
Feb 02 09:30:21 xoa xo-server[1178375]:     created: '20240202T13:57:21Z',
Feb 02 09:30:21 xoa xo-server[1178375]:     finished: '20240202T14:30:21Z',
Feb 02 09:30:21 xoa xo-server[1178375]:     status: 'failure',
Feb 02 09:30:21 xoa xo-server[1178375]:     resident_on: 'OpaqueRef:a482c281-bc5c-6aed-e5cc-91d43c02dc46',
Feb 02 09:30:21 xoa xo-server[1178375]:     progress: 1,
Feb 02 09:30:21 xoa xo-server[1178375]:     type: '<none/>',
Feb 02 09:30:21 xoa xo-server[1178375]:     result: '',
Feb 02 09:30:21 xoa xo-server[1178375]:     error_info: [
Feb 02 09:30:21 xoa xo-server[1178375]:       'IMPORT_ERROR',
Feb 02 09:30:21 xoa xo-server[1178375]:       'INTERNAL_ERROR: [ Unix.Unix_error(Unix.ENOSPC, "write", "") ]'
Feb 02 09:30:21 xoa xo-server[1178375]:     ],
Feb 02 09:30:21 xoa xo-server[1178375]:     other_config: { object_creation: 'complete' },
Feb 02 09:30:21 xoa xo-server[1178375]:     subtask_of: 'OpaqueRef:NULL',
Feb 02 09:30:21 xoa xo-server[1178375]:     subtasks: [],
Feb 02 09:30:21 xoa xo-server[1178375]:     backtrace: '(((process xapi)(filename lib/backtrace.ml)(line 210))((process xapi)(filename ocaml/xapi/import.ml)(line 2131))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 72)))'
Feb 02 09:30:21 xoa xo-server[1178375]:   }
Feb 02 09:30:21 xoa xo-server[1178375]: }
                                  1 Reply Last reply Reply Quote 1
                                  • A Offline
                                    archw @florent
                                    last edited by archw

                                    @florent said in Import from VMware fails after upgrade to XOA 5.91:

                                    XO install folder

                                    Where is the "XO install folder"? I did an ssh into the xoa vm and I don't see what you refer to?
                                    I tried running "git checkout fix_xva_import_thin" but it says "fatal: not a git repository (or any of the parent directories): .git"

                                    florentF 1 Reply Last reply Reply Quote 0
                                    • florentF Offline
                                      florent Vates 🪐 XO Team @archw
                                      last edited by

                                      @archw said in Import from VMware fails after upgrade to XOA 5.91:

                                      @florent said in Import from VMware fails after upgrade to XOA 5.91:

                                      XO install folder

                                      Where is the "XO install folder"? I did an ssh into the xoa vm and I don't see what you refer to?
                                      I tried running "git checkout fix_xva_import_thin" but it says "fatal: not a git repository (or any of the parent directories): .git"

                                      if it's an XOA, even a starter/free one, open a support tunnel, I will take care of it
                                      If not, how did you install your XO from source ?

                                      A 2 Replies Last reply Reply Quote 0
                                      • A Offline
                                        archw @florent
                                        last edited by

                                        @florent
                                        The one I showed above what the starter version. Since I made that post, I also tried running it on the one I built from source and it is version "Master, commit 09247". It has the same errors.

                                        I just opened a support tunnel.

                                        1 Reply Last reply Reply Quote 0
                                        • A Offline
                                          archw @florent
                                          last edited by

                                          @florent I just sent you tunnell id

                                          florentF 1 Reply Last reply Reply Quote 0
                                          • florentF Offline
                                            florent Vates 🪐 XO Team @archw
                                            last edited by

                                            @archw said in Import from VMware fails after upgrade to XOA 5.91:

                                            @florent I just sent you tunnell id

                                            the patch is applied

                                            A 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post