@julien-f haven't seen OOM's after this so looks good, thank you.

@isener just run /opt/xo/xo-server/dist/recover-account-cli.mjs admin@admin.net

Yup, installed nodejs version is beyond LTS version, which could be the cause. It is probably installed manually outside of the script as well since it installs 16.x if missing/too old but doesn’t do anything if version is above.

@florent thanks, yeah this seems to fix it again.

Did some further testing if amount of DNS queries would correlate to the amount of actual connections made to the host. This doesn't seem to be the case which is even more interesting Some results below.

Ran an incremental from delta backup which took in total of 9 minutes:

• Amount of DNS queries: close to 7k
• Amount of HTTPS connects logged to host IP-address: 478.
• Amount of HTTPS connects/disconnects logged in total to host IP-address: 955

Connection counts were about the same with installation from dns.lookup branch provided by @julien-f above, without the amount of DNS queries obviously.

That script has a shebang pointing to node (first line), so no need to define it separately, just execute the script directly.

Sure. @Ascar the executable you’re looking for is not called xo-server-logs for the installation done with XenOrchestraInstallerUpdater script. As mentioned above it’s installed beside xo-server as logs-cli.mjs and can be found here (assuming you didn’t change default install location).

$ sudo /opt/xo/xo-server/dist/logs-cli.mjs -h

xo-server-logs --help, -h
…

Help shows command examples as xo-server-logs, but you need to use above executable instead.

You can also easily make xo-server-logs command to work if you wish by running:

sudo ln -s /opt/xo/xo-server/dist/logs-cli.mjs /usr/local/bin/xo-server-logs

But i wouldn’t recommend it if this is needed only once.

It's quite normal that any new package that adds a service user modifies passwd file or some patch could've added/modified such user. Check the passwd backup file located in /etc/passwd- if its modification time matches the time you ran updates. If yes, then you can just diff passwd and passwd- to see what was actually changed.

@isener said in Forgot password for the webgui:

and when I ping, I get:
ping: xo.my-company.net: Name or service not known

what is wrong?

There is no DNS record for xo.my-company.net or you can’t resolve it from wherever you’re trying now.

xo.my-company.net -> which ip address should this be:
of the xcp-neg server or the server where the xen-orchestar is installed?

from where to run this command:
xo-cli --register http://xo.my-company.net/api/admin@admin.net
the xcp-neg server or the server where the xen-orchestar is installed?

Answer to both would be where XO is installed.

@isener said in Forgot password for the webgui:

I read somewhere that yiu have to use this command in CLI
xo-server-recover-account youremail@here.com

but how to install CLI?

This does not mean you need xo-cli installed. CLI refers to just active shell to your XOA VM, via console or ssh. Just run the command in the shell. If you have XO from sources, there's no such command alias and you need to run the script directly from where your XO is installed, like <installdir>/packages/xo-server/dist/recover-account-cli.mjs

I'm running daily installation from sources on multiple different OS's. All have the same specs: 2vCPU/4GB RAM. This has worked flawlessly for a long time. Recently (starting from 4th/5th Nov) i've started to see OOM errors almost daily during yarn build which then cause it to fail with following error:

Using polyfills: No polyfills were added, since the `useBuiltIns` option was not set.
[01:23:25] Finished 'copyAssets' after 36 s
[01:24:33] Finished 'buildScripts' after 1.73 min
[01:24:33] Finished 'build' after 1.73 min
✖ 1
error Command failed with exit code 1.

It isn't consistent, sometimes it's debian that fails, sometimes ubuntu, sometimes centos/almalinux and so on. Something has definitely changed in the build procedure that eats more RAM than it used to.

I’m fine with increasing the RAM if needed. Just wanted to point this out if there’s something out of the ordinary with latest changes.

How did you install your XO from sources? Manually following the docs or using some 3rd party script?

Anyway the error kinda gives a hint "only root can use...", so you're running XO with a non privileged user right? Did you define useSudo configuration option and add proper sudoers configuration as the documentation suggests? That error comes directly from OS's mount command so you either need to run the process as root (which you're not doing properly if you get this error) or add the config option to use sudo when mounting remotes.

That script has a shebang pointing to node (first line), so no need to define it separately, just execute the script directly.

Sure. @Ascar the executable you’re looking for is not called xo-server-logs for the installation done with XenOrchestraInstallerUpdater script. As mentioned above it’s installed beside xo-server as logs-cli.mjs and can be found here (assuming you didn’t change default install location).

$ sudo /opt/xo/xo-server/dist/logs-cli.mjs -h

xo-server-logs --help, -h
…

Help shows command examples as xo-server-logs, but you need to use above executable instead.

You can also easily make xo-server-logs command to work if you wish by running:

sudo ln -s /opt/xo/xo-server/dist/logs-cli.mjs /usr/local/bin/xo-server-logs

But i wouldn’t recommend it if this is needed only once.

@isener just run /opt/xo/xo-server/dist/recover-account-cli.mjs admin@admin.net

@isener said in Forgot password for the webgui:

and when I ping, I get:
ping: xo.my-company.net: Name or service not known

what is wrong?

There is no DNS record for xo.my-company.net or you can’t resolve it from wherever you’re trying now.

xo.my-company.net -> which ip address should this be:
of the xcp-neg server or the server where the xen-orchestar is installed?

from where to run this command:
xo-cli --register http://xo.my-company.net/api/admin@admin.net
the xcp-neg server or the server where the xen-orchestar is installed?

Answer to both would be where XO is installed.

@isener said in Forgot password for the webgui:

I read somewhere that yiu have to use this command in CLI
xo-server-recover-account youremail@here.com

but how to install CLI?

This does not mean you need xo-cli installed. CLI refers to just active shell to your XOA VM, via console or ssh. Just run the command in the shell. If you have XO from sources, there's no such command alias and you need to run the script directly from where your XO is installed, like <installdir>/packages/xo-server/dist/recover-account-cli.mjs

@julien-f haven't seen OOM's after this so looks good, thank you.

I'm running daily installation from sources on multiple different OS's. All have the same specs: 2vCPU/4GB RAM. This has worked flawlessly for a long time. Recently (starting from 4th/5th Nov) i've started to see OOM errors almost daily during yarn build which then cause it to fail with following error:

Using polyfills: No polyfills were added, since the `useBuiltIns` option was not set.
[01:23:25] Finished 'copyAssets' after 36 s
[01:24:33] Finished 'buildScripts' after 1.73 min
[01:24:33] Finished 'build' after 1.73 min
✖ 1
error Command failed with exit code 1.

It isn't consistent, sometimes it's debian that fails, sometimes ubuntu, sometimes centos/almalinux and so on. Something has definitely changed in the build procedure that eats more RAM than it used to.

I’m fine with increasing the RAM if needed. Just wanted to point this out if there’s something out of the ordinary with latest changes.

Settings -> audit. Assuming you have it enabled. If not, nowhere afaik.

Increasing VM disk size from XCP-ng center/XO only makes the block device larger, partition and filesystem (what you see in df output) won’t resize automatically, you need to do it manually inside the VM.

There’s usually little to no reasons to expose any mgmt systems to internet in actual production environments. Especially if it’s strictly for internal mgmt purposes. XO is no exception. Not because the system would be unsecure, but you simply want to make any attack surface as small as possible. It’s just a best a practice.