@isener just run /opt/xo/xo-server/dist/recover-account-cli.mjs admin@admin.net

@julien-f haven't seen OOM's after this so looks good, thank you.

Yup, installed nodejs version is beyond LTS version, which could be the cause. It is probably installed manually outside of the script as well since it installs 16.x if missing/too old but doesn’t do anything if version is above.

@florent thanks, yeah this seems to fix it again.

Did some further testing if amount of DNS queries would correlate to the amount of actual connections made to the host. This doesn't seem to be the case which is even more interesting Some results below.

Ran an incremental from delta backup which took in total of 9 minutes:

• Amount of DNS queries: close to 7k
• Amount of HTTPS connects logged to host IP-address: 478.
• Amount of HTTPS connects/disconnects logged in total to host IP-address: 955

Connection counts were about the same with installation from dns.lookup branch provided by @julien-f above, without the amount of DNS queries obviously.

That script has a shebang pointing to node (first line), so no need to define it separately, just execute the script directly.

Sure. @Ascar the executable you’re looking for is not called xo-server-logs for the installation done with XenOrchestraInstallerUpdater script. As mentioned above it’s installed beside xo-server as logs-cli.mjs and can be found here (assuming you didn’t change default install location).

$ sudo /opt/xo/xo-server/dist/logs-cli.mjs -h

xo-server-logs --help, -h
…

Help shows command examples as xo-server-logs, but you need to use above executable instead.

You can also easily make xo-server-logs command to work if you wish by running:

sudo ln -s /opt/xo/xo-server/dist/logs-cli.mjs /usr/local/bin/xo-server-logs

But i wouldn’t recommend it if this is needed only once.

It's quite normal that any new package that adds a service user modifies passwd file or some patch could've added/modified such user. Check the passwd backup file located in /etc/passwd- if its modification time matches the time you ran updates. If yes, then you can just diff passwd and passwd- to see what was actually changed.

@isener said in Forgot password for the webgui:

and when I ping, I get:
ping: xo.my-company.net: Name or service not known

what is wrong?

There is no DNS record for xo.my-company.net or you can’t resolve it from wherever you’re trying now.

xo.my-company.net -> which ip address should this be:
of the xcp-neg server or the server where the xen-orchestar is installed?

from where to run this command:
xo-cli --register http://xo.my-company.net/api/admin@admin.net
the xcp-neg server or the server where the xen-orchestar is installed?

Answer to both would be where XO is installed.

@isener said in Forgot password for the webgui:

I read somewhere that yiu have to use this command in CLI
xo-server-recover-account youremail@here.com

but how to install CLI?

This does not mean you need xo-cli installed. CLI refers to just active shell to your XOA VM, via console or ssh. Just run the command in the shell. If you have XO from sources, there's no such command alias and you need to run the script directly from where your XO is installed, like <installdir>/packages/xo-server/dist/recover-account-cli.mjs

I'm running daily installation from sources on multiple different OS's. All have the same specs: 2vCPU/4GB RAM. This has worked flawlessly for a long time. Recently (starting from 4th/5th Nov) i've started to see OOM errors almost daily during yarn build which then cause it to fail with following error:

Using polyfills: No polyfills were added, since the `useBuiltIns` option was not set.
[01:23:25] Finished 'copyAssets' after 36 s
[01:24:33] Finished 'buildScripts' after 1.73 min
[01:24:33] Finished 'build' after 1.73 min
✖ 1
error Command failed with exit code 1.

It isn't consistent, sometimes it's debian that fails, sometimes ubuntu, sometimes centos/almalinux and so on. Something has definitely changed in the build procedure that eats more RAM than it used to.

I’m fine with increasing the RAM if needed. Just wanted to point this out if there’s something out of the ordinary with latest changes.

Increasing VM disk size from XCP-ng center/XO only makes the block device larger, partition and filesystem (what you see in df output) won’t resize automatically, you need to do it manually inside the VM.

Sounds like a client side error if it happens with two different remotes on different systems, which probably explains why there’s nothing in server side logs.

Which OS is XO running? Could this be related: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=949394

Could you try mv on smb share without XO involved to verify it’s not related? Sure it should report other than success if such renaming fails.

I can reproduce this (ref: https://github.com/ronivay/XenOrchestraInstallerUpdater/issues/133) and it seems to be caused by commit: https://github.com/vatesfr/xen-orchestra/commit/49890a09b7a3c4bcbe7f462577c6d0a7da8b7006

If you start from fresh (no redis database specifically) and install from that commit, or latest master, the default user creation doesn't work properly and above mentioned error is present in xo-server logs. It's actually all user creations (their passwords to be exact), not just default user. Doesn't affect existing users so if updating from previous version, everything seemingly works fine until new user is created and login with it fails.

If installing from commit: https://github.com/vatesfr/xen-orchestra/commit/dfce56cee8023ce0dc9292e0c1dda5e6937910e5 with fresh redis database, all works as it should. Also if broken admin@admin.net user was already created with installation from newer commit and redis database wasn't cleared, resetting user password with recover-account-cli.mjs in installation from this commit brings it back to life again.

rjlan created this issue in ronivay/XenOrchestraInstallerUpdater

closed invalid credentials on login to fresh install #133

0 julien-f committed to vatesfr/xen-orchestra

chore: update dev deps

0 julien-f committed to vatesfr/xen-orchestra

feat(async-each): add basic JsDoc typing

@julien-f this changed the situation from thousands of queries in minutes to no noticeable spike in query graphs during backup job, so huge improvement.

Could very well be. With multiple hosts and simultaneous backups, the amount of requests can easily be overwhelming if it multiplies in such scenario. I've actually created an issue about stalled backup due to failed DNS resolution back in 2019 (https://github.com/vatesfr/xen-orchestra/issues/4122). I didn't dig this deep into it then and kinda assumed it was something specific to my environment and switched to using just IP-address. Switched back to domain name at some point. Personally haven't seen failed backups due to same reason recently so could be it retries nowadays at least to some extent.

/etc/hosts sure is also an option but as a static config it takes away the benefit you get from using DNS.

ronivay created this issue in vatesfr/xen-orchestra

closed Failed name resolution stalling backup #4122

Thanks. This seems to fix it for me at least. XO logs UNEXPECTED RESPONSE and FALLBACK for those HVM VM's and console opens up fine.

Locate the packages/xo-server directory to your from sources built xen orchestra instance. There should be a file in dist/recover-account-cli.mjs, copy this file as dist/recover-account-cli.mjs and then run it from the dist directory ./recover-account-cli.mjs admin@something.tld, it'll ask password and either reset the existing users password or create a new one with admin privileges if there isn't one with the email address.

Plugin configuration has good examples for those attributes. There really is no tutorial for your specific environment, depends how your AD is setup. If you have other applications setup with ldap, try to look for hints from their configuration. Note that it does not sync users, they’re being created on successful login after plugin is configured properly and activated.

That "VM" is a snapshot of that running XOA - xbackup1 machine. Open that VM in Xen Orchestra and check snapshots tab, you'll see that the name and uuid (copy button next to snapshot) matches the results from command line.

You can ignore snapshots in listing with: xe vm-list is-a-snapshot=false