Int Error when trying delta backups in XO

kevdog

I keep getting this error message when trying to do delta backups within XO
Error: INTERNAL_ERROR(Storage_interface.Internal_error("(Sys_error "Broken pipe")"))
Start: Jun 19, 2019, 11:29:53 AM
End: Jun 19, 2019, 11:29:53 AM
Duration: a few seconds
Error: INTERNAL_ERROR(Storage_interface.Internal_error("(Sys_error "Broken pipe")"))

I'm not sure what that means. The delta backups are stored to a freenas systme, however I don't even get to the transfer part of this procedure.

DustinB

@kevdog Is the connection to your remote active and connected?

olivierlambert

Please provide more info. XenServer or XCP-ng version? What's your SR type?

kevdog

Xcp-ng version 7.6. SR type is NFS share to FreeNAS with 33Tb of freespace. When checking the connection to the server through the XO interface (Settings->Remotes) it states that the remote appears to work correctly.

Here is my log file:

cloud.getResourceCatalog
{}
{
  "message": "invalid status closed, expected open",
  "name": "ConnectionError",
  "stack": "ConnectionError: invalid status closed, expected open
    at JsonRpcWebSocketClient._assertStatus (/opt/xen-orchestra/node_modules/jsonrpc-websocket-client/src/websocket-client.js:142:12)
    at JsonRpcWebSocketClient.send (/opt/xen-orchestra/node_modules/jsonrpc-websocket-client/src/websocket-client.js:129:9)
    at Peer.<anonymous> (/opt/xen-orchestra/node_modules/jsonrpc-websocket-client/src/index.js:50:11)
    at emitOne (events.js:116:13)
    at Peer.emit (events.js:211:7)
    at Peer.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/src/configure.js:93:16)
    at Peer.push (/opt/xen-orchestra/node_modules/json-rpc-peer/src/index.js:197:13)
    at /opt/xen-orchestra/node_modules/json-rpc-peer/src/index.js:142:11
    at Promise._execute (/opt/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)
    at Promise._resolveFromExecutor (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:488:18)
    at new Promise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)
    at Peer.request (/opt/xen-orchestra/node_modules/json-rpc-peer/src/index.js:139:11)
    at JsonRpcWebSocketClient.call (/opt/xen-orchestra/node_modules/jsonrpc-websocket-client/src/index.js:66:22)
    at /opt/xen-orchestra/packages/xo-server-cloud/src/index.js:70:40
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/opt/xen-orchestra/packages/xo-server-cloud/dist/index.js:16:103)
    at _next (/opt/xen-orchestra/packages/xo-server-cloud/dist/index.js:18:194)
    at /opt/xen-orchestra/packages/xo-server-cloud/dist/index.js:18:364
    at Promise._execute (/opt/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)
    at Promise._resolveFromExecutor (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:488:18)
    at new Promise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)
    at /opt/xen-orchestra/packages/xo-server-cloud/dist/index.js:18:97
    at XoServerCloud._getCatalog (/opt/xen-orchestra/packages/xo-server-cloud/src/index.js:69:22)
    at Xo.getResourceCatalog (/opt/xen-orchestra/packages/xo-server-cloud/src/index.js:23:42)
    at /opt/xen-orchestra/packages/xo-server/src/xo-mixins/api.js:281:32
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/api.js:38:103)
    at _next (/opt/xen-orchestra/packages/xo-server/dist/xo-mixins/api.js:40:194)
    at tryCatcher (/opt/xen-orchestra/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:517:31)
    at Promise._settlePromise (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:574:18)
    at Promise._settlePromiseCtx (/opt/xen-orchestra/node_modules/bluebird/js/release/promise.js:611:10)
    at _drainQueueStep (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:142:12)
    at _drainQueue (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:131:9)
    at Async._drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:147:5)
    at Immediate.Async.drainQueues (/opt/xen-orchestra/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:810:20)
    at tryOnImmediate (timers.js:768:5)
    at processImmediate [as _immediateCallback] (timers.js:745:5)"
}

olivierlambert

This is not the right log (on the last post).

Also I'm not sure to understand, are you on XOA or XO from the sources?

kevdog

XO from source.

Sorry here is the log:

pool.listMissingPatches
{
  "host": "d4271565-f224-4679-8b58-6b67195fb8ce"
}
{
  "errno": "ECONNRESET",
  "code": "ECONNRESET",
  "syscall": "write",
  "url": "https://10.0.1.10/jsonrpc",
  "call": {
    "method": "host.call_plugin",
    "params": [
      "OpaqueRef:9cce4d13-6b29-4204-ab8a-ee4f788f6ce8",
      "updater.py",
      "check_update",
      {}
    ]
  },
  "message": "write ECONNRESET",
  "name": "Error",
  "stack": "Error: write ECONNRESET
    at WriteWrap.afterWrite [as oncomplete] (net.js:868:14)"
}

olivierlambert

I don't understand why are you using a plugin that's not meant for the sources. How did you installed it exactly? Following our official doc? https://xen-orchestra.com/docs/from_the_sources.html
The error message you are showing me is not related to delta backup but to patching. However, it might be interesting: it seems you have network issues. Is your XO in the same network than the host or behind a NAT?

kevdog

FreeNAS is located within same subnet and within LAN as XO. Everything being run in local environment. In terms of plugins -- I'm not sure -- I just use the plugins that came with the XO installation.

olivierlambert

@kevdog no plugins are bundled with XO installation in our official doc. You didn't answer: did you use our official doc to install it or a 3rd party script?

Danp

@kevdog It sounds like you used a 3rd party script to install XO. If you go into Settings > Plugins, you can disable the cloud plugin as I believe it is only used by XOA (the pre-built appliance).

Speaking of XOA, one method to help isolate the issue is to install a copy of XOA and try to connect to your storage system from there. If that works, then we now know that there is something amiss in your XO installation.

Lastly, why did you settle on FreeNAS for your storage? I've read elsewhere that it isn't recommended to use it. Just curious.

olivierlambert

I never heard any issue using FreeNAS as NFS storage for XCP-ng.

Danp

@olivierlambert It doesn't have anything to do with XCP-ng. Just avoid FreeNAS for everything was the mindset. See the link for details.

olivierlambert

The thread is huge, can you summarize the key points?

Danp

Use FreeBSD instead of FreeNAS
Avoid JPE