I asked about this in the Discord server a little while ago. To be honest, one of my employers would love Flex vGPU support. We've been needing to update both our general compute and engineering VDIs so we've been looking at some solutions.

While NVIDIA vGPU pricing isn't horrible, it requires us to have VMware infrastructure which is significantly more expensive (obviously). So we'd really like NVIDIA and/or Intel vGPU support, but understand that it isn't just a few lines of code that needs written and is a very small subset of XCP-ng users. Especially with the alleged uncertainty of Intel right now, it may not even be worth developing anything for their cards, but that's personal conjecture.

I've been building my own VDI solution for XO(A), but that's been stalled for a little bit due to burnout, but I plan on finishing a prototype solution and writing a setup guide in a few weeks. So if you want something for just general compute, keep an eye out!

@hansve Since you have shared storage, you should be able to use the "Start on..." feature under the Advanced tab of the VM. That should let you start it on any of the hosts. I'm curious if it will work though because your NIC isn't being detected. You may have to get a USB Ethernet adapter to try and get it to move.

The other option would be to forget the VMs if you can (similar to VMware's "remove from inventory" option). Then create a VM without any disks attached to it, then add the already existing disk to the VM and start it.

I suppose you could do both of them as well. You'd get a USB NIC so you can have networking to the pool (as not to screw with the XAPI DB), then unregister the VMs, create shell VMs, attach disks, then start.

If none of these work, let me know!

@johnd Can you make sure the networks show as "Connected" on the host you're trying to migrate to? I've run into a few issues where I can't migrate VMs after a host reboot because the networks aren't connected

@SudoOracle Can you try recreating the backup job? It won't clean up the previous backups, but it may kick off a new process. Also, (I'm sure you are, but just double checking) are you on the latest XO(A)?

@itnok I'm sure you already have, but have you ensured that everything in this guide was done?

@Danp

1. The file is getting removed every time, that's why I was questioning the behavior of the command (whether it is grabbing all records or cached records).

2. We can do that, but if the user is deleted, that user entry would be gone.

@gtuminauskas Hello! You can edit the template settings by hovering over Home -> Templates -> Find the template you want to change -> click on the 3 lines (menu) button all the way to the right of the template -> click on the field you want to change (in your case RAM), and change the value!

@McHenry If I'm understanding this correctly, Client Site A, B, n... will each have a server and all share multiple instances at OVH for DR. First off, I hope you give the person that's doing the networking for this a raise .

Second, are you and/or a team the sole manager of the infrastructure? If clients have their own IT team that assists/uses XO(A), this will be more difficult. For the sake of this solution, I'm going to assume that the client can use XO(A) to start VMs in case of a DR scenario.

If I were to build it out, I would build an XO host in OVH and utilize the XO-proxy and the Self Service feature. This way, you can register the XO Proxy to all of the client sites, and manage who can see what VMs, networking/VLANS, and storage.

If I'm wrong about an assumption or solution, let me know!

@Danp To execute the command, it's almost instant. To download the file, the same 4ish minutes. I'd also like to note that despite holding a similar 10,000 records, the file becomes significantly longer each run when running the unfiltered command (about 5-10 MiB larger, currently at 610 MiB).

I guess I have three questions now:

• Is there any way to make my filter "more efficient"?
• When executing the command with and without the filter, is it getting audit records that have since been purged from XO(A)? If so, that's great news for me. Just means that the command takes more time, but I don't have to worry about getting and filtering those records separately.
• As @jgrafton just brought up to me: Is it possible to store the username of the user that made the snapshot in the VM snapshot object? We're basically combining the above audit command and joining it on the snapshot uuid of xo-cli list-objects type=VM-snapshot in order to get username of the person who did the snapshot.

Before last week, xo-cli audit.getRecords --json took maybe 30-60 seconds to run. It's weird that it just started in the middle of the week unprovoked.

@julien-f Thank you for the info! After some playing around and some outside help, this is what worked: xo-cli rest --json get -o records.json plugins/audit/records fields=data,event,subject,time,id filter=data:method:vm.snapshot. However, anytime I run it, it takes about 4 minutes for that command to run (albeit now only downloading about 20 KiB). And with this python, it's going to loop through each XO(A) instance, so some guidance on how to lower this time (or troubleshoot) would be appreciated!

EDIT: The downloading part only takes a second or two, but waiting for the command to execute takes 3-4 minutes

@julien-f

After some playing around, xo-cli rest --json get -o records.json plugins/audit/records fields=data,event,subject,time,id actually got the records and put them in JSON format. However, I don't need all the records, is there a way to match the 'method' to 'vm.snapshot'? That way I'm not downloading and parsing a 500ish MiB file every time ? I've tried adding method=vm.snapshot and event=vm.snapshot, but neither worked (I'm probably putting it in the wrong spot)

@julien-f It finished with a records.ndjson file of 510.31 MiB. There's been the max 10,000 audit records since I've started, so I can't imagine the size is part of the issue. And yes, the REST API would work, but it was a bit more elegant to have the json loaded in memory rather than get the file first, then load it. Not that it makes a huge difference.

However, now that I'm taking a look at it, all of the "method" fields show "host.isPubKeyTooShort". I'll have to sift through the data to see if everything I need from the records are there.

@olivierlambert That is the output, it times out after a few minutes then spits that out

Howdy! I'm getting the error below in XO when trying to run xo-cli audit.getRecords --json in a python script. However, this same error happens when running the command locally. The XO instance registers successfully, and the other commands work as expected. This started happening last week, and I've since updated XO and the VM it's running on (and yes, I did turn it off and back on again ).

I was thinking it could be the self-signed cert, but every other command works, and the --allowUnauthorized flag is set in the register command and is reflected in the error message. If anyone has an idea or something for me to check, please let me know!

node:events:510
    throw err; // Unhandled 'error' event
    ^

Error [ERR_UNHANDLED_ERROR]: Unhandled error. (ErrorEvent {
  [Symbol(kTarget)]: <ref *1> WebSocket {
    _events: [Object: null prototype] {
      close: [Function],
      error: [Function],
      message: [Function]
    },
    _eventsCount: 3,
    _maxListeners: undefined,
    _binaryType: 'nodebuffer',
    _closeCode: 1006,
    _closeFrameReceived: false,
    _closeFrameSent: false,
    _closeMessage: <Buffer >,
    _closeTimer: Timeout {
      _idleTimeout: 30000,
      _idlePrev: [TimersList],
      _idleNext: [TimersList],
      _idleStart: 204363,
      _onTimeout: [Function: bound ],
      _timerArgs: undefined,
      _repeat: null,
      _destroyed: false,
      [Symbol(refed)]: true,
      [Symbol(kHasPrimitive)]: false,
      [Symbol(asyncId)]: 11785,
      [Symbol(triggerId)]: 11782
    },
    _errorEmitted: true,
    _extensions: { 'permessage-deflate': [PerMessageDeflate] },
    _paused: false,
    _protocol: '',
    _readyState: 2,
    _receiver: Receiver {
      _events: [Object],
      _writableState: [WritableState],
      _maxListeners: undefined,
      _allowSynchronousEvents: true,
      _binaryType: 'nodebuffer',
      _extensions: [Object],
      _isServer: false,
      _maxPayload: 104857600,
      _skipUTF8Validation: false,
      _bufferedBytes: 0,
      _buffers: [],
      _compressed: true,
      _payloadLength: 68954412,
      _mask: undefined,
      _fragmented: 0,
      _masked: false,
      _fin: true,
      _opcode: 1,
      _totalPayloadLength: 68954412,
      _messageLength: 0,
      _fragments: [],
      _errored: false,
      _loop: false,
      _state: 5,
      _eventsCount: 6,
      [Symbol(shapeMode)]: true,
      [Symbol(kCapture)]: false,
      [Symbol(websocket)]: [Circular *1]
    },
    _sender: Sender {
      _extensions: [Object],
      _socket: [TLSSocket],
      _firstFragment: true,
      _compress: true,
      _bufferedBytes: 0,
      _queue: [],
      _state: 0,
      onerror: [Function: senderOnError],
      [Symbol(websocket)]: [Circular *1]
    },
    _socket: TLSSocket {
      _tlsOptions: [Object],
      _secureEstablished: true,
      _securePending: false,
      _newSessionPending: false,
      _controlReleased: true,
      secureConnecting: false,
      _SNICallback: null,
      servername: '<REDACTED>',
      alpnProtocol: false,
      authorized: false,
      authorizationError: 'DEPTH_ZERO_SELF_SIGNED_CERT',
      encrypted: true,
      _events: [Object: null prototype],
      _eventsCount: 5,
      connecting: false,
      _hadError: false,
      _parent: null,
      _host: '<REDACTED>',
      _closeAfterHandlingError: false,
      _readableState: [ReadableState],
      _writableState: [WritableState],
      allowHalfOpen: false,
      _maxListeners: undefined,
      _sockname: null,
      _pendingData: null,
      _pendingEncoding: '',
      server: undefined,
      _server: null,
      ssl: [TLSWrap],
      _requestCert: true,
      _rejectUnauthorized: false,
      parser: null,
      _httpMessage: null,
      timeout: 0,
      [Symbol(alpncallback)]: null,
      [Symbol(res)]: [TLSWrap],
      [Symbol(verified)]: true,
      [Symbol(pendingSession)]: null,
      [Symbol(async_id_symbol)]: 74,
      [Symbol(kHandle)]: [TLSWrap],
      [Symbol(lastWriteQueueSize)]: 8,
      [Symbol(timeout)]: null,
      [Symbol(kBuffer)]: null,
      [Symbol(kBufferCb)]: null,
      [Symbol(kBufferGen)]: null,
      [Symbol(shapeMode)]: true,
      [Symbol(kCapture)]: false,
      [Symbol(kSetNoDelay)]: true,
      [Symbol(kSetKeepAlive)]: false,
      [Symbol(kSetKeepAliveInitialDelay)]: 0,
      [Symbol(kBytesRead)]: 0,
      [Symbol(kBytesWritten)]: 0,
      [Symbol(connect-options)]: [Object],
      [Symbol(websocket)]: [Circular *1]
    },
    _bufferedAmount: 0,
    _isServer: false,
    _redirects: 0,
    _autoPong: true,
    _url: 'wss://<REDACTED>/api/',
    _req: null,
    [Symbol(shapeMode)]: false,
    [Symbol(kCapture)]: false
  },
  [Symbol(kType)]: 'error',
  [Symbol(kError)]: Error: invalid code lengths set
      at genericNodeError (node:internal/errors:984:15)
      at wrappedFn (node:internal/errors:538:14)
      at Zlib.zlibOnError [as onerror] (node:zlib:190:17) {
    errno: -3,
    code: 'Z_DATA_ERROR',
    [Symbol(status-code)]: 1007
  },
  [Symbol(kMessage)]: 'invalid code lengths set'
})
    at Xo.emit (node:events:508:17)
    at WebSocket.<anonymous> (/opt/xo/xo-builds/xen-orchestra-202410241844/node_modules/jsonrpc-websocket-client/dist/websocket-client.js:171:18)
    at callListener (/opt/xo/xo-builds/xen-orchestra-202410241844/node_modules/ws/lib/event-target.js:290:14)
    at WebSocket.onError (/opt/xo/xo-builds/xen-orchestra-202410241844/node_modules/ws/lib/event-target.js:230:9)
    at WebSocket.emit (node:events:519:28)
    at Receiver.receiverOnError (/opt/xo/xo-builds/xen-orchestra-202410241844/node_modules/ws/lib/websocket.js:1199:15)
    at Receiver.emit (node:events:519:28)
    at emitErrorNT (node:internal/streams/destroy:169:8)
    at emitErrorCloseNT (node:internal/streams/destroy:128:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  code: 'ERR_UNHANDLED_ERROR',
  context: ErrorEvent {
    [Symbol(kTarget)]: <ref *2> WebSocket {
      _events: [Object: null prototype] {
        close: [Function: onClose] {
          [Symbol(kIsForOnEventAttribute)]: false,
          [Symbol(kListener)]: [Function: bound _onClose]
        },
        error: [Function: onError] {
          [Symbol(kIsForOnEventAttribute)]: false,
          [Symbol(kListener)]: [Function (anonymous)]
        },
        message: [Function: onMessage] {
          [Symbol(kIsForOnEventAttribute)]: false,
          [Symbol(kListener)]: [Function (anonymous)]
        }
      },
      _eventsCount: 3,
      _maxListeners: undefined,
      _binaryType: 'nodebuffer',
      _closeCode: 1006,
      _closeFrameReceived: false,
      _closeFrameSent: false,
      _closeMessage: Buffer(0) [Uint8Array] [],
      _closeTimer: <ref *1> Timeout {
        _idleTimeout: 30000,
        _idlePrev: TimersList {
          _idleNext: [Circular *1],
          _idlePrev: [Circular *1],
          expiry: 234363,
          id: -9007199254740175,
          msecs: 30000,
          priorityQueuePosition: 2
        },
        _idleNext: TimersList {
          _idleNext: [Circular *1],
          _idlePrev: [Circular *1],
          expiry: 234363,
          id: -9007199254740175,
          msecs: 30000,
          priorityQueuePosition: 2
        },
        _idleStart: 204363,
        _onTimeout: [Function: bound ],
        _timerArgs: undefined,
        _repeat: null,
        _destroyed: false,
        [Symbol(refed)]: true,
        [Symbol(kHasPrimitive)]: false,
        [Symbol(asyncId)]: 11785,
        [Symbol(triggerId)]: 11782
      },
      _errorEmitted: true,
      _extensions: {
        'permessage-deflate': PerMessageDeflate {
          _maxPayload: 104857600,
          _options: {},
          _threshold: 1024,
          _isServer: false,
          _deflate: DeflateRaw {
            _writeState: [Uint32Array],
            _events: [Object],
            _readableState: [ReadableState],
            _writableState: [WritableState],
            allowHalfOpen: true,
            _maxListeners: undefined,
            _eventsCount: 2,
            bytesWritten: 211,
            _handle: [Zlib],
            _outBuffer: [Buffer [Uint8Array]],
            _outOffset: 160,
            _chunkSize: 16384,
            _defaultFlushFlag: 0,
            _finishFlushFlag: 4,
            _defaultFullFlushFlag: 3,
            _info: undefined,
            _maxOutputLength: 4294967296,
            _level: -1,
            _strategy: 0,
            [Symbol(shapeMode)]: true,
            [Symbol(kCapture)]: false,
            [Symbol(kCallback)]: null,
            [Symbol(kError)]: null,
            [Symbol(total-length)]: 0,
            [Symbol(buffers)]: [],
            [Symbol(callback)]: null
          },
          _inflate: null,
          params: [Object: null prototype] {}
        }
      },
      _paused: false,
      _protocol: '',
      _readyState: 2,
      _receiver: Receiver {
        _events: {
          close: undefined,
          error: [Function: receiverOnError],
          prefinish: undefined,
          finish: undefined,
          drain: [Function: receiverOnDrain],
          conclude: [Function: receiverOnConclude],
          message: [Function: receiverOnMessage],
          ping: [Function: receiverOnPing],
          pong: [Function: receiverOnPong]
        },
        _writableState: WritableState {
          highWaterMark: 16384,
          length: 0,
          corked: 0,
          onwrite: [Function: bound onwrite],
          writelen: 0,
          bufferedIndex: 0,
          pendingcb: 1,
          [Symbol(kState)]: 17582526,
          [Symbol(kBufferedValue)]: null,
          [Symbol(kWriteCbValue)]: null,
          [Symbol(kErroredValue)]: Error: invalid code lengths set
              at genericNodeError (node:internal/errors:984:15)
              at wrappedFn (node:internal/errors:538:14)
              at Zlib.zlibOnError [as onerror] (node:zlib:190:17) {
            errno: -3,
            code: 'Z_DATA_ERROR',
            [Symbol(status-code)]: 1007
          }
        },
        _maxListeners: undefined,
        _allowSynchronousEvents: true,
        _binaryType: 'nodebuffer',
        _extensions: {
          'permessage-deflate': PerMessageDeflate {
            _maxPayload: 104857600,
            _options: {},
            _threshold: 1024,
            _isServer: false,
            _deflate: [DeflateRaw],
            _inflate: null,
            params: [Object: null prototype] {}
          }
        },
        _isServer: false,
        _maxPayload: 104857600,
        _skipUTF8Validation: false,
        _bufferedBytes: 0,
        _buffers: [],
        _compressed: true,
        _payloadLength: 68954412,
        _mask: undefined,
        _fragmented: 0,
        _masked: false,
        _fin: true,
        _opcode: 1,
        _totalPayloadLength: 68954412,
        _messageLength: 0,
        _fragments: [],
        _errored: false,
        _loop: false,
        _state: 5,
        _eventsCount: 6,
        [Symbol(shapeMode)]: true,
        [Symbol(kCapture)]: false,
        [Symbol(websocket)]: [Circular *2]
      },
      _sender: Sender {
        _extensions: {
          'permessage-deflate': PerMessageDeflate {
            _maxPayload: 104857600,
            _options: {},
            _threshold: 1024,
            _isServer: false,
            _deflate: [DeflateRaw],
            _inflate: null,
            params: [Object: null prototype] {}
          }
        },
        _socket: <ref *3> TLSSocket {
          _tlsOptions: {
            allowHalfOpen: undefined,
            pipe: false,
            secureContext: [SecureContext],
            isServer: false,
            requestCert: true,
            rejectUnauthorized: false,
            session: undefined,
            ALPNProtocols: undefined,
            requestOCSP: undefined,
            enableTrace: undefined,
            pskCallback: undefined,
            highWaterMark: undefined,
            onread: undefined,
            signal: undefined
          },
          _secureEstablished: true,
          _securePending: false,
          _newSessionPending: false,
          _controlReleased: true,
          secureConnecting: false,
          _SNICallback: null,
          servername: '<REDACTED>',
          alpnProtocol: false,
          authorized: false,
          authorizationError: 'DEPTH_ZERO_SELF_SIGNED_CERT',
          encrypted: true,
          _events: [Object: null prototype] {
            close: [Array],
            end: [Array],
            error: [Function: socketOnError],
            newListener: [Array],
            connect: undefined,
            secure: [Function: onConnectSecure],
            data: undefined,
            drain: undefined
          },
          _eventsCount: 5,
          connecting: false,
          _hadError: false,
          _parent: null,
          _host: '<REDACTED>',
          _closeAfterHandlingError: false,
          _readableState: ReadableState {
            highWaterMark: 16384,
            buffer: [Array],
            bufferIndex: 0,
            length: 15395,
            pipes: [],
            awaitDrainWriters: null,
            [Symbol(kState)]: 110635268
          },
          _writableState: WritableState {
            highWaterMark: 16384,
            length: 8,
            corked: 0,
            onwrite: [Function: bound onwrite],
            writelen: 8,
            bufferedIndex: 0,
            pendingcb: 1,
            [Symbol(kState)]: 118259972,
            [Symbol(kBufferedValue)]: null,
            [Symbol(kWriteCbValue)]: [Function (anonymous)]
          },
          allowHalfOpen: false,
          _maxListeners: undefined,
          _sockname: null,
          _pendingData: null,
          _pendingEncoding: '',
          server: undefined,
          _server: null,
          ssl: TLSWrap {
            _parent: [TCP],
            _parentWrap: null,
            _secureContext: [SecureContext],
            reading: true,
            onkeylog: [Function: onkeylog],
            onhandshakestart: [Function: noop],
            onhandshakedone: [Function (anonymous)],
            onocspresponse: [Function: onocspresponse],
            onnewsession: [Function: onnewsessionclient],
            onerror: [Function: onerror],
            [Symbol(owner_symbol)]: [Circular *3]
          },
          _requestCert: true,
          _rejectUnauthorized: false,
          parser: null,
          _httpMessage: null,
          timeout: 0,
          [Symbol(alpncallback)]: null,
          [Symbol(res)]: TLSWrap {
            _parent: [TCP],
            _parentWrap: null,
            _secureContext: [SecureContext],
            reading: true,
            onkeylog: [Function: onkeylog],
            onhandshakestart: [Function: noop],
            onhandshakedone: [Function (anonymous)],
            onocspresponse: [Function: onocspresponse],
            onnewsession: [Function: onnewsessionclient],
            onerror: [Function: onerror],
            [Symbol(owner_symbol)]: [Circular *3]
          },
          [Symbol(verified)]: true,
          [Symbol(pendingSession)]: null,
          [Symbol(async_id_symbol)]: 74,
          [Symbol(kHandle)]: TLSWrap {
            _parent: [TCP],
            _parentWrap: null,
            _secureContext: [SecureContext],
            reading: true,
            onkeylog: [Function: onkeylog],
            onhandshakestart: [Function: noop],
            onhandshakedone: [Function (anonymous)],
            onocspresponse: [Function: onocspresponse],
            onnewsession: [Function: onnewsessionclient],
            onerror: [Function: onerror],
            [Symbol(owner_symbol)]: [Circular *3]
          },
          [Symbol(lastWriteQueueSize)]: 8,
          [Symbol(timeout)]: null,
          [Symbol(kBuffer)]: null,
          [Symbol(kBufferCb)]: null,
          [Symbol(kBufferGen)]: null,
          [Symbol(shapeMode)]: true,
          [Symbol(kCapture)]: false,
          [Symbol(kSetNoDelay)]: true,
          [Symbol(kSetKeepAlive)]: false,
          [Symbol(kSetKeepAliveInitialDelay)]: 0,
          [Symbol(kBytesRead)]: 0,
          [Symbol(kBytesWritten)]: 0,
          [Symbol(connect-options)]: {
            rejectUnauthorized: false,
            ciphers: 'TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!SRP:!CAMELLIA',
            checkServerIdentity: [Function: checkServerIdentity],
            minDHSize: 1024,
            allowSynchronousEvents: true,
            autoPong: true,
            protocolVersion: 13,
            maxPayload: 104857600,
            skipUTF8Validation: false,
            perMessageDeflate: true,
            followRedirects: false,
            maxRedirects: 10,
            socketPath: undefined,
            hostname: undefined,
            protocol: undefined,
            timeout: undefined,
            method: 'GET',
            host: '<REDACTED>',
            path: undefined,
            port: 443,
            createConnection: [Function: tlsConnect],
            defaultPort: 443,
            headers: [Object],
            _defaultAgent: [Agent],
            servername: '<REDACTED>',
            singleUse: true
          },
          [Symbol(websocket)]: [Circular *2]
        },
        _firstFragment: true,
        _compress: true,
        _bufferedBytes: 0,
        _queue: [],
        _state: 0,
        onerror: [Function: senderOnError],
        [Symbol(websocket)]: [Circular *2]
      },
      _socket: <ref *3> TLSSocket {
        _tlsOptions: {
          allowHalfOpen: undefined,
          pipe: false,
          secureContext: SecureContext { context: SecureContext {} },
          isServer: false,
          requestCert: true,
          rejectUnauthorized: false,
          session: undefined,
          ALPNProtocols: undefined,
          requestOCSP: undefined,
          enableTrace: undefined,
          pskCallback: undefined,
          highWaterMark: undefined,
          onread: undefined,
          signal: undefined
        },
        _secureEstablished: true,
        _securePending: false,
        _newSessionPending: false,
        _controlReleased: true,
        secureConnecting: false,
        _SNICallback: null,
        servername: '<REDACTED>',
        alpnProtocol: false,
        authorized: false,
        authorizationError: 'DEPTH_ZERO_SELF_SIGNED_CERT',
        encrypted: true,
        _events: [Object: null prototype] {
          close: [
            [Function: onSocketCloseDestroySSL],
            [Function: socketOnClose]
          ],
          end: [ [Function: onReadableStreamEnd], [Function: socketOnEnd] ],
          error: [Function: socketOnError],
          newListener: [ [Function: keylogNewListener], [Function: newListener] ],
          connect: undefined,
          secure: [Function: onConnectSecure],
          data: undefined,
          drain: undefined
        },
        _eventsCount: 5,
        connecting: false,
        _hadError: false,
        _parent: null,
        _host: '<REDACTED>',
        _closeAfterHandlingError: false,
        _readableState: ReadableState {
          highWaterMark: 16384,
          buffer: [
            [Buffer [Uint8Array]],
            [Buffer [Uint8Array]],
            [Buffer [Uint8Array]],
            [Buffer [Uint8Array]],
            [Buffer [Uint8Array]],
            [Buffer [Uint8Array]],
            [Buffer [Uint8Array]]
          ],
          bufferIndex: 0,
          length: 15395,
          pipes: [],
          awaitDrainWriters: null,
          [Symbol(kState)]: 110635268
        },
        _writableState: WritableState {
          highWaterMark: 16384,
          length: 8,
          corked: 0,
          onwrite: [Function: bound onwrite],
          writelen: 8,
          bufferedIndex: 0,
          pendingcb: 1,
          [Symbol(kState)]: 118259972,
          [Symbol(kBufferedValue)]: null,
          [Symbol(kWriteCbValue)]: [Function (anonymous)]
        },
        allowHalfOpen: false,
        _maxListeners: undefined,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: undefined,
        _server: null,
        ssl: TLSWrap {
          _parent: TCP {
            reading: [Getter/Setter],
            onconnection: null,
            [Symbol(owner_symbol)]: [Circular *3]
          },
          _parentWrap: null,
          _secureContext: SecureContext { context: SecureContext {} },
          reading: true,
          onkeylog: [Function: onkeylog],
          onhandshakestart: [Function: noop],
          onhandshakedone: [Function (anonymous)],
          onocspresponse: [Function: onocspresponse],
          onnewsession: [Function: onnewsessionclient],
          onerror: [Function: onerror],
          [Symbol(owner_symbol)]: [Circular *3]
        },
        _requestCert: true,
        _rejectUnauthorized: false,
        parser: null,
        _httpMessage: null,
        timeout: 0,
        [Symbol(alpncallback)]: null,
        [Symbol(res)]: TLSWrap {
          _parent: TCP {
            reading: [Getter/Setter],
            onconnection: null,
            [Symbol(owner_symbol)]: [Circular *3]
          },
          _parentWrap: null,
          _secureContext: SecureContext { context: SecureContext {} },
          reading: true,
          onkeylog: [Function: onkeylog],
          onhandshakestart: [Function: noop],
          onhandshakedone: [Function (anonymous)],
          onocspresponse: [Function: onocspresponse],
          onnewsession: [Function: onnewsessionclient],
          onerror: [Function: onerror],
          [Symbol(owner_symbol)]: [Circular *3]
        },
        [Symbol(verified)]: true,
        [Symbol(pendingSession)]: null,
        [Symbol(async_id_symbol)]: 74,
        [Symbol(kHandle)]: TLSWrap {
          _parent: TCP {
            reading: [Getter/Setter],
            onconnection: null,
            [Symbol(owner_symbol)]: [Circular *3]
          },
          _parentWrap: null,
          _secureContext: SecureContext { context: SecureContext {} },
          reading: true,
          onkeylog: [Function: onkeylog],
          onhandshakestart: [Function: noop],
          onhandshakedone: [Function (anonymous)],
          onocspresponse: [Function: onocspresponse],
          onnewsession: [Function: onnewsessionclient],
          onerror: [Function: onerror],
          [Symbol(owner_symbol)]: [Circular *3]
        },
        [Symbol(lastWriteQueueSize)]: 8,
        [Symbol(timeout)]: null,
        [Symbol(kBuffer)]: null,
        [Symbol(kBufferCb)]: null,
        [Symbol(kBufferGen)]: null,
        [Symbol(shapeMode)]: true,
        [Symbol(kCapture)]: false,
        [Symbol(kSetNoDelay)]: true,
        [Symbol(kSetKeepAlive)]: false,
        [Symbol(kSetKeepAliveInitialDelay)]: 0,
        [Symbol(kBytesRead)]: 0,
        [Symbol(kBytesWritten)]: 0,
        [Symbol(connect-options)]: {
          rejectUnauthorized: false,
          ciphers: 'TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!SRP:!CAMELLIA',
          checkServerIdentity: [Function: checkServerIdentity],
          minDHSize: 1024,
          allowSynchronousEvents: true,
          autoPong: true,
          protocolVersion: 13,
          maxPayload: 104857600,
          skipUTF8Validation: false,
          perMessageDeflate: true,
          followRedirects: false,
          maxRedirects: 10,
          socketPath: undefined,
          hostname: undefined,
          protocol: undefined,
          timeout: undefined,
          method: 'GET',
          host: '<REDACTED>',
          path: undefined,
          port: 443,
          createConnection: [Function: tlsConnect],
          defaultPort: 443,
          headers: {
            'Sec-WebSocket-Version': 13,
            'Sec-WebSocket-Key': '/6jaAonjgPC7ytTPeX/epA==',
            Connection: 'Upgrade',
            Upgrade: 'websocket',
            'Sec-WebSocket-Extensions': 'permessage-deflate; client_max_window_bits'
          },
          _defaultAgent: Agent {
            _events: [Object: null prototype],
            _eventsCount: 2,
            _maxListeners: undefined,
            defaultPort: 443,
            protocol: 'https:',
            options: [Object: null prototype],
            requests: [Object: null prototype] {},
            sockets: [Object: null prototype] {},
            freeSockets: [Object: null prototype] {},
            keepAliveMsecs: 1000,
            keepAlive: true,
            maxSockets: Infinity,
            maxFreeSockets: 256,
            scheduling: 'lifo',
            maxTotalSockets: Infinity,
            totalSocketCount: 0,
            maxCachedSessions: 100,
            _sessionCache: [Object],
            [Symbol(shapeMode)]: false,
            [Symbol(kCapture)]: false
          },
          servername: '<REDACTED>',
          singleUse: true
        },
        [Symbol(websocket)]: [Circular *2]
      },
      _bufferedAmount: 0,
      _isServer: false,
      _redirects: 0,
      _autoPong: true,
      _url: 'wss://<REDACTED>/api/',
      _req: null,
      [Symbol(shapeMode)]: false,
      [Symbol(kCapture)]: false
    },
    [Symbol(kType)]: 'error',
    [Symbol(kError)]: Error: invalid code lengths set
        at genericNodeError (node:internal/errors:984:15)
        at wrappedFn (node:internal/errors:538:14)
        at Zlib.zlibOnError [as onerror] (node:zlib:190:17) {
      errno: -3,
      code: 'Z_DATA_ERROR',
      [Symbol(status-code)]: 1007
    },
    [Symbol(kMessage)]: 'invalid code lengths set'
  }
}

Node.js v20.18.0

That's what it was. For whatever reason, ubuntu defaults to installing these versions for node and npm respectively:
v12.22.9
8.5.1

Following the steps here fixed it. As always, thanks!!

Howdy!

I'm trying to install xo-cli on a new Ubuntu 22.02 VM. I don't need the full XO, just the cli. I've installed NPM, and did sudo npm install -g xo-cli, but when I try to run xo-cli register..., it returns ths:

internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'fs' imported from /usr/local/lib/node_modules/xo-cli/index.mjs
    at packageResolve (internal/modules/esm/resolve.js:664:9)
    at moduleResolve (internal/modules/esm/resolve.js:705:18)
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:798:11)
    at Loader.resolve (internal/modules/esm/loader.js:100:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:246:28)
    at ModuleWrap.<anonymous> (internal/modules/esm/module_job.js:47:40)
    at link (internal/modules/esm/module_job.js:46:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Am I missing something I need to install? I've had this happen before on a Rocky 9 VM as well, but someone else had fixed that for me.

@Tristis-Oris Do you have HA enabled? Also, I believe that maintenance mode is supposed to be automatically disabled after a reboot

@infecticide I won't speak for the folks at Vates, but in my (and by extension my employer's) experience, they do! And they treat you just like a regular paying customer. They've been super prompt and resolve issues faster than VMware!

@marcom The only thing that I would add (and I'm guessing you knew, adding this just in case) is to restore the backup of XO(A) and the pool metadata.

@tony And again, double check the BIOS updates. There may be a bug fix in one of them for Xen