Unable to start VM, xenopsd Memory error

[d1rtym0nk3y]

A colleague was having issues with a VM and rebooted it via the XO ui.
Now it fails to start with the error message below.

I have tried a few things

• "Start On" various nodes in the cluster, same error
• toolstack restart on the pool master
• xe vm-reset-powerstate force=true uuid=...

But so far, same error everytime I try to start the VM.
Has anyone seen this error before can point towards a possible cause ?

vm.start
{
  "id": "d0ede814-9dd5-a932-c540-3070a03f8c72",
  "bypassMacAddressesCheck": false,
  "force": false
}
{
  "code": "INTERNAL_ERROR",
  "params": [
    "xenopsd internal error: Memory_interface.Memory_error([S(Internal_error);S((Sys_error \"Broken pipe\"))])"
  ],
  "call": {
    "method": "VM.start",
    "params": [
      "OpaqueRef:a4169698-c3e4-4bd8-ac0b-e4d826d9ce3b",
      false,
      false
    ]
  },
  "message": "INTERNAL_ERROR(xenopsd internal error: Memory_interface.Memory_error([S(Internal_error);S((Sys_error \"Broken pipe\"))]))",
  "name": "XapiError",
  "stack": "XapiError: INTERNAL_ERROR(xenopsd internal error: Memory_interface.Memory_error([S(Internal_error);S((Sys_error \"Broken pipe\"))]))
    at Function.wrap (/usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/_XapiError.js:16:12)
    at /usr/local/lib/node_modules/xo-server/node_modules/xen-api/src/transports/json-rpc.js:35:27
    at AsyncResource.runInAsyncScope (async_hooks.js:197:9)
    at cb (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/util.js:355:42)
    at tryCatcher (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/usr/local/lib/node_modules/xo-server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (internal/timers.js:464:21)
    at process.topLevelDomainCallback (domain.js:147:15)
    at process.callbackTrampoline (internal/async_hooks.js:129:24)"
}

[d1rtym0nk3y]

setting static max = dynamic min = dynamic max did not fix it

a toolstack restart on all hosts in the cluster seems to have resolved the issue, I have not had a chance to dig in to find a root cause yet

[olivierlambert]

Never heard of this one before. Try to check if you have memory static max = dynamic min = dynamic max.

[d1rtym0nk3y]

setting static max = dynamic min = dynamic max did not fix it

a toolstack restart on all hosts in the cluster seems to have resolved the issue, I have not had a chance to dig in to find a root cause yet

[olivierlambert]

Weird. Feel free to give more feedback if it happens again