I thought I did that.
Oh I see whats happening. My cold moves are freaking out.
The first time, I restarted the toolstack for the server I was working on the # of tasks dropped by 50% or so. as soon as I tried my cold move of a VM again, im up to 50-60 tasks again. Going from Xen 7.1 to XCP 7.6. and a Xeon 2630 V3 to a 2620 V0. Which I know is a bad idea, hence the cold move.
@olivierlambert
control+shift+R on that page and they all come back.
I even tried to kill them from the cli on any given xen server, and they don't seem to go away.
xe task-cancel force=true uuid=
Haven't used my XO in a few weeks, but went in today, and see something like 50-60 tasks all like the below. Is this normal? Haven't noticed before. How can I get rid of?
Looks like it took me to build 37cdbc19e
xo-server 5.101.0
xo-web 5.102.0
and making new VM works.
and to think I just happened to need to make a new one after updating the other day, wasn't really looking for a bug.
went back to my previous build (using the XenOrchestraInstallerUpdater)
xo-server 5.96.0/ xo-web 5.97.2 build c4839 and new VM worked fine.
XCP-ng Center had no problem making new VM, by the way.
Can't create new VM today, after updating to 5.101.0 / 5.102.0 build 281a1
Any thoughts on this? Am I missing something?
thanks.
vm.create
{
"clone": true,
"existingDisks": {},
"installation": {
"method": "cdrom",
"repository": "a05463cd-8a9f-40e5-9087-8f1ddda0c750"
},
"name_label": "Windows Server 2019 (64-bit)",
"template": "9788e5a6-c4ec-a4ba-2083-c448722a5974-d986ea98-09fa-4e9a-9ee6-e6c78119496f",
"VDIs": [
{
"bootable": false,
"device": "0",
"size": 268435456000,
"type": "system",
"SR": "000988d9-5c09-4c63-86a1-8cb904d0642f",
"name_description": "Created by XO",
"name_label": "Windows Server 2019 (64-bit)_oleri"
}
],
"VIFs": [
{
"network": "46843b23-dd90-2b78-0f9a-dfaac7847e9c",
"allowedIpv4Addresses": [],
"allowedIpv6Addresses": []
}
],
"coresPerSocket": 2,
"CPUs": "4",
"cpusMax": 4,
"cpuWeight": null,
"cpuCap": null,
"name_description": "",
"memory": 17179869184,
"bootAfterCreate": true,
"copyHostBiosStrings": false,
"secureBoot": false,
"share": false,
"coreOs": false,
"tags": [],
"hvmBootFirmware": "uefi"
}
{
"code": 10,
"data": {
"errors": [
{
"instancePath": "/VDIs/0",
"schemaPath": "#/properties/VDIs/items/additionalProperties",
"keyword": "additionalProperties",
"params": {
"additionalProperty": "bootable"
},
"message": "must NOT have additional properties"
},
{
"instancePath": "/VDIs/0",
"schemaPath": "#/properties/VDIs/items/additionalProperties",
"keyword": "additionalProperties",
"params": {
"additionalProperty": "device"
},
"message": "must NOT have additional properties"
},
{
"instancePath": "/VDIs/0",
"schemaPath": "#/properties/VDIs/items/additionalProperties",
"keyword": "additionalProperties",
"params": {
"additionalProperty": "name_description"
},
"message": "must NOT have additional properties"
},
{
"instancePath": "/VDIs/0",
"schemaPath": "#/properties/VDIs/items/additionalProperties",
"keyword": "additionalProperties",
"params": {
"additionalProperty": "name_label"
},
"message": "must NOT have additional properties"
}
]
},
"message": "invalid parameters",
"name": "XoError",
"stack": "XoError: invalid parameters
at Module.invalidParameters (/opt/xo/xo-builds/xen-orchestra-202209011436/packages/xo-common/api-errors.js:26:11)
at Xo.call (file:///opt/xo/xo-builds/xen-orchestra-202209011436/packages/xo-server/src/xo-mixins/api.mjs:65:20)
at Api.#callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202209011436/packages/xo-server/src/xo-mixins/api.mjs:390:19)"
}
Updated to Master build, a66ae, xo-server 5.92.0, xo-web 5.96.0.
Consoles seem to work fine again.
thanks all!
@florent
I ran the install upgrade script from @ronivay with the fix_console_proxy_xcp_7x branch. Seems to be working.
I don't know if this helps:
most of my VMs are Windows. There are a few Linux, Centos 7+, Rocky, and Ubuntu 18.
the production XO I use, is behind an Apache reverse proxy, using https. The temporary one, is just XO, using http on 8080. Looks like all of these are PVHVM, (so, HVM?) it claims. Both of these XO's are VMs on the same host, and the other hosts are on the same subnet, but some are on a different VLAN. There are also a number that are in a co-location facility, connected via a point to point fiber link, on a different VLAN, but all behind our edge router.
Console working: XCP 8.1, XS 7.1
Console not working: XCP 7.6 - I can't say for sure that these have any updates applied, though.
@olivierlambert
hrm, well, I bumped my temp install of XO back to the master branch, and consoles broke. -Appears- to only broke with XCP 7.6. 7.1, sites worked.
@olivierlambert , I received exact instructions from @ronivay on how to do that. Using commit b9ff3db9b in my new temporary install of XO, everything works fine.
Moving to c99120b seems to break the console on anything other than XCP 8.2.
And I've mentioned before I think: all of our hardware does support XCP 8.2, but almost none of the RAID card monitor software works there. 80% of them are using 3Ware/ LSI 9750's. I can't recall the exact problem, but the executable for the web-admin doesn't run.
after googling some git commands; im not very versed in git at all, so not sure how to get that. 
this is the one that last worked for me, and the one im currently using. I save the last 3 in @ronivay script.
v5.16.0-8953-g04913cabb
this is the one that I pulled and it seems to break:
v5.16.0-9058-gf6e1b9571
I see another comment on the issue tracker from @ronivay. I'll try to get to his suggestion a little later.
and just to link these two together for someone in the way future:
https://github.com/ronivay/XenOrchestraInstallerUpdater/issues/116
Made another VM with Rocky Linux 8, and a copy of the existing to duplicate my set up. Added some hosts to it, and sure enough same problem. Tried to pull an update to XenOrchestraInstallerUpdater, but its already up to date, and the author says its auto-updates.
So far, not finding a common thread, between them showing up and not showing up. Happens in Firefox, and Chrome, Doesn't appear tob e related to Xen Guest Tools version, getting device drivers from MS or no, or version of XenServer.
As long as im on this old version of XO, it seems to work.
I do see these when i bring up a VM console that doesn't show, and DON'T see them when one does show.
Failed when connecting: Failed to connect to server ( (code: 1005)) util.js:44:54
Error util.js:44
_fail rfb.js:578
f rfb.js:233
onclose websock.js:305
open websock.js:307
_connect rfb.js:384
_updateConnectionState rfb.js:546
connect rfb.js:279
p react-novnc.js:106
componentDidMount react-novnc.js:111
React 12
listen EventListener.js:29
React 29
Tried changing state of a disconnected RFB object util.js:44:54
Error util.js:44
_updateConnectionState rfb.js:473
disconnect rfb.js:284
_clean react-novnc.js:66
p react-novnc.js:72
(Async: setTimeout handler)
_onUpdateState react-novnc.js:44
_updateConnectionState rfb.js:519
_fail rfb.js:588
f rfb.js:233
onclose websock.js:305
open websock.js:307
_connect rfb.js:384
_updateConnectionState rfb.js:546
connect rfb.js:279
p react-novnc.js:106
componentDidMount react-novnc.js:111
React 12
listen EventListener.js:29
React 29
trapBubbledEvent
listenTo
z
_updateDOMProperties
mountComponent
mountComponent
mountChildren
_createInitialChildren
mountComponent
mountComponent
mountChildren
_createInitialChildren
mountComponent
mountComponent
performInitialMount
mountComponent
mountComponent
performInitialMount
mountComponent
mountComponent
performInitialMount
mountComponent
mountComponent
performInitialMount
mountComponent
mountComponent
performInitialMount
mountComponent
mountComponent
Is there any reason this might happen if my Xen Server is Citrix 7.1, or XCP 7.6? This is the current suspicion, after a suggestion from the writer of the install script.
Updated to a newer Xen orchestra recently, didn't record what version. But noticed today, that the VM console for a LOT of VMs is broken, and doesn't show up. XCP-ng Center still works fine.
Tried restarting the toolstack on one host, made no difference. Use the XenOrchestraInstallerUpdater to go to update, today, and it didn't fix anything Went back to the earliest version I had, that I can roll back to, and the consoles seem to work again. The working version is: build 04913 5.89/ 5.94.
Any ideas?
Ahh! Alarms on the Dashboard page...
Xen XCP Center has this Notifications tab. its real handy, and displays info like the below, but Xen Orchestra doesn't, that I can tell. I may be missing something.
@benjireis
So..... I ran
journalctl -u xo-server -f -n 50 on XO machine
and then created a netwokr, and got the same second error, not the first one. and it created the network on the XCP 7.6 machine. They both seem to have the sdn service running, but only the 8.2 machine has a firewall rule for 6640.
do I need one for the XO machine as well? Im not completely sure, but I think im cool other than the weird (erroneous?) messages.
Nov 19 10:17:36 xo1.logistics.int xo-server[654246]: 2021-11-19T17:17:36.827Z xo:xo-server:sdn-controller INFO Private network registered { privateNetwork: '42b5f6da-e06a-4801-84ae-[redacted]' }
Nov 19 10:17:36 xo1.logistics.int xo-server[654246]: 2021-11-19T17:17:36.889Z xo:xo-server:sdn-controller INFO New network created {
Nov 19 10:17:36 xo1.logistics.int xo-server[654246]: privateNetwork: '42b5f6da-e06a-4801-84ae-[redacted]',
Nov 19 10:17:36 xo1.logistics.int xo-server[654246]: network: 'temp',
Nov 19 10:17:36 xo1.logistics.int xo-server[654246]: pool: ''
Nov 19 10:17:36 xo1.logistics.int xo-server[654246]: }
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: 2021-11-19T17:17:39.700Z xo:xo-server:sdn-controller:tls-connect ERROR TLS connection failed {
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: error: Error: Client network socket disconnected before secure TLS connection was established
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at connResetException (internal/errors.js:639:14)
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at TLSSocket.onConnectEnd (_tls_wrap.js:1570:19)
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at TLSSocket.emit (events.js:412:35)
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at TLSSocket.emit (domain.js:475:12)
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at TLSSocket.patchedEmit [as emit] (/opt/xo/xo-builds/xen-orchestra-202111011545/@xen-orchestra/log/configure.js:118:17)
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at endReadableNT (internal/streams/readable.js:1334:12)
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: at processTicksAndRejections (internal/process/task_queues.js:82:21) {
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: code: 'ECONNRESET',
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: path: undefined,
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: host: '172.16.[redacted]',
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: port: 6640,
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: localAddress: undefined
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: },
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: address: '172.16.[redacted]',
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: port: 6640
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: }
Nov 19 10:17:39 xo1.logistics.int xo-server[654246]: 2021-11-19T17:17:39.703Z xo:api WARN bberndt | sdnController.createPrivateNetwork(...) [3s] =!> Error: Client network socket disconnected before secure TLS connection was established
I went back to my XCP-ng 8.2 pool. It looks like its been creating these networks all along, despite the error.
So, I guess the question remains why the error on XCP-ng 7.6? (somewhat rhetorical, I know its old, and outa support)