Ran XOA updater, got 'CANNOT GET /', web UI has stopped working.

waveguide

Just today, I deployed XOA using the Vates online tools which setup a VM on my XCP-ng host. The XOA version date was I believe form April 2024. I want to test the v6 UI so I registered XOA and ran the updater which seemed to work for a while, and then appeared to pause.

After that point, if I tried to reach any page on XOA I would get 'CANNOT GET /'. I restarted the VM and now I'm unable to load any page at all.

I was able to access the console for the XOA. If I run systemctl status xo-sever.service I get the following:

x xo-server.service - XO Server
Loaded: loaded (/etc/systemd/system/xo-server.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since (date)
Duration: 4ms
Process: 1135 ExecStart=/usr/local/bin/xo-server (code-exited, status=203/EXEC)
Main PID: 1135 (code=exited, status=203/EXEC)
CPU: 735us

Date xoa systemd[1]: xo-server.service: Scheduled restart job, restart counter is at 5.
Date xoa systemd[1]: Stopped xo-server.service - XO Service.
Date xoa systemd[1]: xo-server.service: Start request repeated too quickly.
Date xoa systemd[1]: xo-server.service: Failed with result 'exit-code'.
Date xoa systemd[1]: Failed to start xo-server.service - XO Server.

I tried deleting this VM, redeployed, and ran the updater again. Exact same issue. Update appears to pause - this time I noticed it seemed to happen around "installing packages" which was right after updating node and npm.

I found this other post here with a similar issue from 5 years ago, but I don't understand the fix suggested or if that would apply.
: https://xcp-ng.org/forum/topic/1472/cannot-get-error-after-update

This feels like a bug or error if the XOA that's available cannot run the updater without breaking the web UI.

If it helps at all, the node version is now v18.19.1 and the npm version is 10.2.4.

Danp

@waveguide I just performed a new deployment of XOA and it registered and upgraded as expected.

Did you perhaps interrupt the update process? What happens if you run xoa-updater --upgrade again?

waveguide

@Danp I ran the update utility from the XOA web UI and didn't touch anything, so I am assuming there wasn't an interruption.

Do I need to know what folder xoa-updater is in? Because this is the result.

ssh xoa@MYIPADDRESS
xoa@MYIPADDRESS's password:
Linux xoa 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64
 __   __             ____           _               _
 \ \ / /            / __ \         | |             | |
  \ V / ___ _ __   | |  | |_ __ ___| |__   ___  ___| |_ _ __ __ _
   > < / _ \ '_ \  | |  | | '__/ __| '_ \ / _ \/ __| __| '__/ _` |
  / . \  __/ | | | | |__| | | | (__| | | |  __/\__ \ |_| | | (_| |
 /_/ \_\___|_| |_|  \____/|_|  \___|_| |_|\___||___/\__|_|  \__,_|

Welcome to XOA Unified Edition, with Pro Support.

* Restart XO: sudo systemctl restart xo-server.service
* Display status: sudo systemctl status xo-server.service
* Display logs: sudo journalctl -u xo-server.service
* Register your XOA: sudo xoa-updater --register
* Update your XOA: sudo xoa-updater --upgrade

OFFICIAL XOA DOCUMENTATION HERE: https://xen-orchestra.com/docs/xoa.html

Support available at https://xen-orchestra.com/#!/member/support

In case of issues, use `xoa check` for a quick health check.

Build number: 24.04.01

Based on Debian GNU/Linux 12 (Stable) 64bits in PVHVM mode

Last login: Fri Aug 30 22:16:55 2024
[19:08 31] xoa@xoa:~$ sudo xoa-updater --upgrade
[sudo] password for xoa:
sudo: xoa-updater: command not found
[19:11 31] xoa@xoa:~$

Per the guidance I also tried xoa check and again got command not found.

Danp

@waveguide I've performed the update via the web interface and the CLI without issues. However, you can use this KB article to correct the missing xoa and xoa-updater.

waveguide

@Danp said in Ran XOA updater, got 'CANNOT GET /', web UI has stopped working.:

@waveguide I've performed the update via the web interface and the CLI without issues. However, you can use this KB article to correct the missing xoa and xoa-updater.

Awesome, we're making progress.

I ran npm i -g --unsafe-perm https://xen-orchestra.com/xoa-cli.tgz and xoa is now working, so I was able to run xoa check.

✔  1/16 - Node version
✔  2/16 - Memory
✖  3/16 - XOA status: Error: spawn xoa-updater ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:284:19)
    at onErrorNT (node:internal/child_process:477:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn xoa-updater',
  path: 'xoa-updater',
  spawnargs: []
}
✔  4/16 - xo-server config syntax
✔  5/16 - Disk space for /
✔  6/16 - Disk space for /var
✔  7/16 - Native SMB support
✔  8/16 - Fetching VM UUID
✔  9/16 - xoa-support user
✖ 10/16 - NTP synchronization: no
✔ 11/16 - XOA version
✔ 12/16 - /var is writable
✔ 13/16 - Appliance registration
✔ 14/16 - local SSH server
✔ 15/16 - npm version
✔ 16/16 - Internet connectivity

So some sort of issue now with xoa-updater. Running systemctl restart xoa-updater has no effect. I checked the status for both xoa-updater and xo-server.service and both still have 203 errors. xoa-updater still returns command not found.

So I have the xoa-cli working, but nothing else yet.

I could try to deploy and update XOA a third time, but I'm not really clear on what to do differently. If there is something wrong with my infrastructure or setup that is letting a failure occur, it would help to be able to check relevant logs and see where that failure happened.

EDIT: I tried npm install and got

npm ERR! code ENOENT
npm ERR! syscall open
npm ERR! path /home/xoa/package.json
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, open '/home/xoa/package.json'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-09-01T01_05_37_135Z-debug-0.log
[21:05 31] xoa:xoa$ cat /root/.npm/_logs/2024-09-01T01_05_37_135Z-debug-0.log

I checked /home/xoa and I have a package-lock.json but no package.json.

Is this the package.json I should have?
https://github.com/vatesfr/xen-orchestra/blob/master/package.json

Danp

@waveguide There are additional commands that you can run to install xoa-updater if it is missing. Please go back and reread the KB article.

waveguide

My apologies @Danp , I had misunderstood the comments on the other commands in that KB article.

Good news, things are working, but it didn't seem like they would be.

xoa run npm i -g --unsafe-perm https://xen-orchestra.com/xoa-updater.tgz Seemed to work. I was then able to run systemctl restart xoa-updater and xoa-updater --upgrade.

After the upgrade command, this is the feedback:

 xoa:xoa$ xoa-updater --upgrade
stable channel selected
Downloading packages...
  xen-orchestra 5.95.2 . ok ✔
  xo-server-telemetry 0.5.0 ...................................... ok ✔
  xen-orchestra-web 0.0.3 ................................................... ok ✔
  xo-server-xoa 0.29.0 .................................................................................................................. ok ✔
  xo-web-free 5.147.0 ................................................................................................................................................................................................................................................................................................................................................................................................................................. ok ✔
  xoa-cli 0.38.2 ...................................................................................................................................................................................................................... ok ✔
  xen-orchestra-upload-ova 0.1.6 ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ ok ✔
  xo-server 5.145.0 ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ok ✔
Following packages will be removed:
  xoa-updater 0.50.0
Removing packages...
Saving current updater version in /usr/local/lib/node_modules/xoa-updater-previous ...ok ✔
  xoa-updater ✔
New versions available:
  node 18.20.2 (18.19.1 installed)
  npm 10.5.0 (10.2.4 installed)
Downloading packages...
  node 18.20.2 ...... ok ✔
  npm 10.5.0 .................................................................................................................................................................. ok ✔
Installing packages...
Upgrading Node/npm failed, re-installing old packages Command failed with exit code 1: npm i -g --unsafe-perm --quiet /tmp/xoa-updater/node-18.20.2.tgz
npm ERR! code 1
npm ERR! path /usr/local/lib/node_modules/node
npm ERR! command failed
npm ERR! command sh -c ./installer
npm ERR! installing : node-v18.20.2
npm ERR! curl: (28) Failed to connect to nodejs.org port 443 after 261014 ms: Couldn't connect to server
npm ERR!
npm ERR!   Error: download preflight failed for '18.20.2' (https://nodejs.org/dist/v18.20.2/node-v18.20.2-linux-x64.tar.xz)

npm ERR! A complete log of this run can be found in: /tmp/.npm/_logs/2024-09-01T16_42_27_685Z-debug-0.log
New packages available:
  xoa-updater 0.50.0
Downloading packages...
Installing packages...
  xoa-updater ✔
  xoa-updater ✔

✖ {
  message: 'Command failed with exit code 1: npm i -g --unsafe-perm --quiet /tmp/xoa-updater/node-18.20.2.tgz\n' +
    'npm ERR! code 1\n' +
    'npm ERR! path /usr/local/lib/node_modules/node\n' +
    'npm ERR! command failed\n' +
    'npm ERR! command sh -c ./installer\n' +
    'npm ERR! installing : node-v18.20.2\n' +
    "npm ERR! curl: (28) Failed to connect to nodejs.org port 443 after 261014 ms: Couldn't connect to server\n" +
    'npm ERR! \n' +
    "npm ERR!   Error: download preflight failed for '18.20.2' (https://nodejs.org/dist/v18.20.2/node-v18.20.2-linux-x64.tar.xz)\n" +
    '\n' +
    'npm ERR! A complete log of this run can be found in: /tmp/.npm/_logs/2024-09-01T16_42_27_685Z-debug-0.log'
}

At this point I thought there was a failure - but then I was able to run systemctl restart xo-server.service and access both the current web UI and the v6 UI.

I'm interested in trying the newer v6 features, so I just changed from the stable channel to latest using the web UI and ran upgrade - but it gave me an error:

9/1/2024, 1:02:48 PM: New versions available:
9/1/2024, 1:02:48 PM:   xoa-updater 0.50.1 (0.50.0 installed)
9/1/2024, 1:02:48 PM: Downloading packages...
9/1/2024, 1:02:49 PM:   xoa-updater 0.50.1 ....................................................................................................................................................................................... ok ✔
9/1/2024, 1:02:49 PM: Installing packages...
9/1/2024, 1:03:02 PM: Saving current updater version in /usr/local/lib/node_modules/xoa-updater-previous ...ok ✔
9/1/2024, 1:03:03 PM:   xoa-updater ✔
9/1/2024, 1:03:14 PM:   xoa-updater ✔
9/1/2024, 1:03:14 PM: Lost connection with xoa-updater
9/1/2024, 1:03:14 PM: xoa-updater could not be reached

Using ssh I went back to the command line and ran xoa-updater --upgrade again and this time it is proceeding forward.

And now it all works! Very weird. I think I'm in good shape now.