error on copy VM cross pools

Gheppy

Hi,
I can't copy VM that is running cross pool with XOCE.
I have:

two pools with Local Storage with over 2Tb available space and min 16 Gb RAM,
and an VM with 50Gb HDD and 4G RAM
2 x XOCE version on ubuntu 19.10 server and ubuntu 18.04 server, source 5.51.1 from Nov 15, 2019

when I try to copy a VM cross pools I get the error below

I compile XCOE on ubuntu 19.10, source 5.51.1 from Oct 29, 2019 and the error is gone.
I complie XOCE on ubuntu 19.10, source 5.51.1 from Oct 30, 2019 and the error reappeared.
Maybe an error in the code?

root@192.168.10.202 Xapi#putResource /import/ missing length
2019-11-15T10:17:21.605Z xo:api WARN admin@admin.net | vm.copy(...) [2s] =!> Error: read ECONNRESET
2019-11-15T10:17:21.620Z xo:xo-server WARN possibly unhandled rejection { error:
   { XapiError: IMPORT_ERROR(INTERNAL_ERROR: [ XML Error: expected root element line 1 ])
    at Function.wrap (/opt/xen-orchestra/packages/xen-api/src/_XapiError.js:16:11)
    at _default (/opt/xen-orchestra/packages/xen-api/src/_getTaskResult.js:11:28)
    at Xapi._addRecordToCache (/opt/xen-orchestra/packages/xen-api/src/index.js:812:37)
    at events.forEach.event (/opt/xen-orchestra/packages/xen-api/src/index.js:833:13)
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (/opt/xen-orchestra/packages/xen-api/src/index.js:823:11)
    at /opt/xen-orchestra/packages/xen-api/src/index.js:984:13
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/opt/xen-orchestra/packages/xen-api/dist/index.js:58:103)
    at _next (/opt/xen-orchestra/packages/xen-api/dist/index.js:60:194)
    at tryCatcher (/opt/xen-orchestra/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:15:14)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)
     code: 'IMPORT_ERROR',
     params:
      [ 'INTERNAL_ERROR: [ XML Error: expected root element line 1 ]' ],
     call: undefined,
     url: 'https://192.168.10.202/import/?sr_id=OpaqueRef%3A3eef5ca2-497c-4226-8329-80a7d0e3af68&session_id=OpaqueRef%3A2957d60c-98f7-4ed4-b190-ec3991e4dd80&task_id=OpaqueRef%3A8bd8c493-213e-4f26-954a-abbd1a24e21b',
     task:
      task {
        uuid: '7be1a1fb-72b6-1e75-d3e9-ce2a5bd93569',
        name_label: '[XO] VM import',
        name_description: '',
        allowed_operations: [],
        current_operations: {},
        created: '20191115T10:17:21Z',
        finished: '20191115T10:17:21Z',
        status: 'failure',
        resident_on: 'OpaqueRef:8dc9f047-feb9-4c11-b5ec-2c8ca2a62975',
        progress: 1,
        type: '<none/>',
        result: '',
        error_info: [Array],
        other_config: {},
        subtask_of: 'OpaqueRef:NULL',
        subtasks: [],
        backtrace: '(((process"xapi @ xen-v2")(filename lib/backtrace.ml)(line 210))((process"xapi @ xen-v2")(filename ocaml/xapi/import.ml)(line 1569))((process"xapi @ xen-v2")(filename ocaml/xapi/server_helpers.ml)(line 80)))' },
     [Symbol(originalCallSite)]:
      [ CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {} ],
     [Symbol(mutatedCallSite)]:
      [ [Object],
        [Object],
        [Object],
        [Object],
        CallSite {},
        [Object],
        [Object],
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {},
        CallSite {} ] } }

olivierlambert

How did you install Xen Orchestra in the first place?

Gheppy

In all cases I followed steps from https://xen-orchestra.com/docs/from_the_sources.html
And I tested on fresh debian 9 OS to.
The errors are from debian OS command line after I run xo-server manualy and then I tried to copy VM, with or without compresion

olivierlambert

Good, can you pick a commit or two behind master to see if it's a problem on XO directly?

Gheppy

I done this with all versions by day, from 30.11.2019 to 04.11.2019 (I take the last version on that day) , and I have the same problems.
The older version are ok.

olivierlambert

Do you have a commit id from when it worked the last time? (or the commit id of the first time it failed)

Gheppy

Last commit ID that it works for me, is:
46da11a52e86deac9bfaecfa147e292cc15a2f0a

After that it doesn't work anymore.
I tested from ID 21dd195b0d063e43213259c071a64d014f317882

olivierlambert

Thx, I'm pinging @pdonias on this

Gheppy

I tested this IDs:
d6e6fa57359fa1722046eaddc315a789249dbd06 - is OK with /without compresion
a47bb1436485cf1e348f4fc3dedfdb7937184b51 - is Ok with /without compresion

fd7d52d38bf4d6ffc6288d0c86acdbbfdaca3fad - NOT OK

Danp

@Gheppy said in error on copy VM cross pools:

fd7d52d38bf4d6ffc6288d0c86acdbbfdaca3fad - NOT OK

This is the same commit that we discussed in following open issue --

https://github.com/vatesfr/xen-orchestra/issues/4640

BluejacketScott created this issue in vatesfr/xen-orchestra

closed XO From source restore backup fails with VDI_IO_ERROR #4640

pdonias

@Danp said in error on copy VM cross pools:

@Gheppy said in error on copy VM cross pools:

fd7d52d38bf4d6ffc6288d0c86acdbbfdaca3fad - NOT OK

This is the same commit that we discussed in following open issue --

https://github.com/vatesfr/xen-orchestra/issues/4640

/cc @julien-f @badrAZ

BluejacketScott created this issue in vatesfr/xen-orchestra

closed XO From source restore backup fails with VDI_IO_ERROR #4640

julien-f

I'm working on it at this very moment, I'll let you know what I find.