Posts made by ITJamie | XCP-ng and XO forum

ITJamie

It would be nice to have be able to be able to exclude certain vms from netbox sync. eg vms tagged with "Continuous Replication"

This would be a similar tag-based chooser to the "Ignored VM tags" on the load-balancer plugin.

Why:
we have indefinite change log retention enabled in netbox. the additions and removals of each replication is chewing through db space quite quickly.
we also just dont need these replicated vms in netbox, just the "original" vm is sufficient for us

ITJamie

Almalinux 8.3/8.4/8.5 templates would be a nice edition to the hub

ITJamie

Ive done a fresh install on debian 11 with node 14.18.3
It now works! from windows and mac. ova's and disk imports

ITJamie

@nraynaud sadly ive started to see import errors again...

Ive tried changing builds on the xoa side but it doesnt seem to fix it...
previous ovf files that were working have not stopped importing.

example of the error on xcp-ng side

Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] Importing raw VDI R:d744d9fee162 failed with exception Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] Raised Server_error(VDI_IO_ERROR, [ Device I/O errors ])
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 1/12 xapi Raised at file ocaml/xapi/vhd_tool_wrapper.ml, line 75
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 2/12 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 3/12 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 4/12 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 5/12 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 6/12 xapi Called from file ocaml/xapi/import_raw_vdi.ml, line 156
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 7/12 xapi Called from file ocaml/xapi/import_raw_vdi.ml, line 194
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 8/12 xapi Called from file ocaml/xapi/server_helpers.ml, line 100
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 9/12 xapi Called from file ocaml/xapi/server_helpers.ml, line 121
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 10/12 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 11/12 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 35
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 12/12 xapi Called from file lib/backtrace.ml, line 177
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace]
Feb 24 15:08:32 xcp-ng-03958 xapi: [debug||1252 /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:aecf708cfad5 created by task D:6567b55f1ad9
Feb 24 15:08:32 xcp-ng-03958 xapi: [ info||1252 /var/lib/xcp/xapi|session.logout D:df9fcc048c68|xapi_session] Session.destroy trackid=ec26a660c6eeea965008c581e6547c0d
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||import] Caught exception in import handler: VDI_IO_ERROR: [ Device I/O errors ]
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] VDI.import D:6567b55f1ad9 failed with exception Unix.Unix_error(Unix.EPIPE, "single_write", "")
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] Raised Unix.Unix_error(Unix.EPIPE, "single_write", "")
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 1/9 xapi Raised at file unix.ml, line 328
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 2/9 xapi Called from file lib/xapi-stdext-unix/unixext.ml, line 469
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 3/9 xapi Called from file lib/xapi-stdext-unix/unixext.ml, line 471
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 4/9 xapi Called from file ocaml/xapi/import_raw_vdi.ml, line 232
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 5/9 xapi Called from file ocaml/xapi/server_helpers.ml, line 100
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 6/9 xapi Called from file ocaml/xapi/server_helpers.ml, line 121
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 7/9 xapi Called from file lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 8/9 xapi Called from file map.ml, line 135
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 9/9 xapi Called from file sexp_conv.ml, line 147
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace]
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] VDI.import D:957854cadc4e failed with exception Unix.Unix_error(Unix.EPIPE, "single_write", "")
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] Raised Unix.Unix_error(Unix.EPIPE, "single_write", "")
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace] 1/1 xapi Raised at file (Thread 1158 has no backtrace table. Was with_backtraces called?, line 0
Feb 24 15:08:32 xcp-ng-03958 xapi: [error||1158 :::80||backtrace]
Feb 24 15:08:32 xcp-ng-03958 xapi: [debug||149 ||xenops] Event on VM 12558119-28e5-4a64-9053-1c0074c5b641; resident_here = true
Feb 24 15:08:32 xcp-ng-03958 xapi: [debug||149 ||dummytaskhelper] task timeboxed_rpc D:168d1d548a5c created by task D:a90e0cbb13bb

i also installed the official appliance on another ip. that worked correctly for loading the same ova files...

I did notice that the official ova image mentions node 14 during its upgrade process. I built from sources with node 17 since the xoa docs say to use latest. Think this issue could be down to using a much newer node version?

ITJamie

@nraynaud
I left the above running until now in firefox. it still had not showed any data in response or request.

Once i refreshed the page I got this console error
importVms watchguard-dimension_2_2.ova TypeError: NetworkError when attempting to fetch resource. index.js:172:159005

This would seem to me that the page had been waiting for something all that time

ITJamie

@nraynaud So this is interesting.
there is no payload at all, and it takes quite awhile for any of the tabs (including headers) to show any data.
the response and request section are completely empty.

heres the "copied as curl"

curl 'http://<ip>/api/xwE65b7EvmOp46LgogDxER7co_8Jl1P2NktYW_oszOs' -X POST -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'Referer: http://<ip>/' -H 'Content-Type: multipart/form-data; boundary=---------------------------274750692132175193903430548728' -H 'Origin: http://<ip>' -H 'Content-Length: 960097718' -H 'Connection: keep-alive' -H 'Cookie: <removed>'

ITJamie

@nraynaud
if there is a way to quickly enable a lot of console logging for this im happy to swap out the xoa build and re-test and give you logs

ITJamie

@olivierlambert
firefox (latest after i upgraded it) on windows just worked!
managed to upload the ubuntu image and the watchguard image!

so nether firefox nor chrome worked on mac.
firefox (97.0) did work on windows.
firefox uploads worked to both local storage and nfs

chrome didnt work on windows when i tried yesterday. however today when i rdp'd in i noticed it was awaiting an upgrade. post upgrade it did manage to upload (im not sure how out of date it was, i dont use windows much)

so currently it means we can work with xoa uploader on windows. but not mac (at least, not m1x arm64 macs, ill try pull out one of my older macs later and see if it works on x86-64 bit mac)

if theres any additional tests you need me to perform to try help that other user @nraynaud mentioned id be happy to.

ITJamie

@ITJamie Screenshot 2022-02-17 at 11.57.00.png

after messing around re-attempting uploads. i managed to get this to appear in debugger console logs...

ITJamie

@olivierlambert local ext and nfs. ive tried both. ive also tried it against different xcp-ng hosts,all installed from the same 8.2.0 iso

tried with firefox. same deal. i noticed in osx's activity monitor it only sent 10MB total

ITJamie

@nraynaud said in ova import failing silently. logs show timeout against xcp-ng node:

Non-zero size required

confirmed i do get the same error. I have not managed to successfully upload any ova or vmdk through the gui

for additional context, i am running this on Version 98.0.4758.80 (Official Build) (arm64) chrome on an m1x mac.
ive also tried it through latest chrome on a windows machine

I saw very little network throughput from the point i pressed "upload" to the point the error appeared in logs. I have a feeling the vmdk is just not being converted in the browser. Can we enable additional logging in the vmdk to vhd code?

ITJamie

@olivierlambert
heres the exact one im importing
https://cloud-images.ubuntu.com/focal/20220215/focal-server-cloudimg-amd64.ova

ITJamie

@olivierlambert nothing shows up in the console log in dev tools in chrome.
i tried it from another pc on our network. same thing.

i got the cli upload command to work from a linux vm. so at least i was able to get unblocked. but would be nice to be able to use the UI for this

ITJamie

XOA / XCP import failing
XOA built from source, master branch. ref: bf194f5c8503e4a09e8f6d84c0619d669e7dd172
Node 17.5.0
Yarn 1.22.17
Debian 11

xcp-ng: 8.2.0. fully patched through XOA and rebooted

I confirmed that the xcp-ng node URL below is reachable from XOA (self signed cert, but it is working)
I saw another forum post about a similar issue https://xcp-ng.org/forum/topic/2853/import-ova-silently-fails. ive added 10gb of ram to the XOA server but it still made no difference.
the xcp-ng server has 8gb of memory assigned to the control domain.

Any ideas?

xoa logs:

2022-02-15T16:03:58.130Z xo:main INFO + WebSocket connection (::ffff:<IP REMOVED>)
2022-02-15T16:09:11.247Z xo:xo-server ERROR unhandled error event {
  error: Error: HTTP connection has timed out
      at IncomingMessage.emitAbortedError (/opt/xen-orchestra/node_modules/http-request-plus/index.js:83:19)
      at Object.onceWrapper (node:events:645:28)
      at IncomingMessage.emit (node:events:526:28)
      at IncomingMessage.emit (node:domain:475:12)
      at IncomingMessage.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:118:17)
      at IncomingMessage._destroy (node:_http_incoming:179:10)
      at _destroy (node:internal/streams/destroy:109:25)
      at IncomingMessage.destroy (node:internal/streams/destroy:71:5)
      at TLSSocket.socketCloseListener (node:_http_client:402:11)
      at TLSSocket.emit (node:events:538:35)
      at TLSSocket.emit (node:domain:475:12)
      at TLSSocket.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:118:17)
      at node:net:687:12
      at TCP.done (node:_tls_wrap:580:7)
      at TCP.callbackTrampoline (node:internal/async_hooks:130:17) {
    canceled: false,
    method: 'PUT',
    url: 'https://<IP REMOVED>/import_raw_vdi/?format=vhd&vdi=OpaqueRef%3Ac3f705d6-773d-4b23-a00e-30948cb0df34&session_id=OpaqueRef%3A035a6aea-bfe7-48a2-9e4f-40c33e6adf65&task_id=OpaqueRef%3A2fd4af46-8ced-46c4-9f68-d63efe15c5f0',
    timeout: true
  }
}

xcp-ng logs (all logs from start to 5 minute timeout): https://pastebin.pl/view/007f62a3

ITJamie

Is there documentation anywhere of the list of available variables for cloud-init/cloud-config templates?
Is it possible to use custom variables from a custom vm template in the cloud-init/cloud-config template?