XO user authentication - excessive and unknown tasks

stevezemlicka

This is very similar to the following post:
https://xcp-ng.org/forum/topic/7865/xo-tasks-xo-user-authentication-too-many

However there are a few differences that may warrant creating a new topic with reference to the old. Like the other topic, I am getting excessive "XO user authentication" tasks. However one difference is, when clicking on the "Open raw log" option, I just receive a "not found" page (not a 404 error but rather a page that simply displays the text "not found") and when I click cancel I receive a dialog from the cancel operation stating "Not Found". Additionally, most the tasks in question have a status of "started" and are yellow rather than green. Finally, I have performed a xcp-ng and xo server reboot with no change.

This is a new and freshly compiled XO instance from last night. xo-server is 5.124.0 and xo-client is 5.126.0. I'm on fully patched XCP-ng 8.2.1.

Did I do something wrong? Is this a known issue? Any thoughts on what might be causing it? Is there additional information I can include? I'm currently tailing some of the logs on the hypervisor including xensource.log but I'm not entirely sure what would be helpful.

XO user authentication tasks.jpg

olivierlambert

Hi,

Are you on the latest commit from the master branch? (the commit number is visible in the About section). If you confirm that, I will ping the right person

stevezemlicka

@olivierlambert
I was originally using commit 1fac7. I reinstalled/upgraded today to 339d9 and the issue is still present. One interesting thing is these "started" tasks disappear when the page is refreshed.

olivierlambert

Ping @julien-f

stevezemlicka

@julien-f
I updated to commit 3410c today and the issue is still present.

julien-f

Hello!

Indeed, it's a know issue, only tasks for failed authentications are kept, successful tasks are removed otherwise they would flood the logs (validating an authentication token, which happens for every requests done to xo-server by xo-web is an authentication).

But the subscription process (used to keep the list up to date on the UI side) currently does not properly handle task removals which can lead to ghost tasks which are still marked as running and leads to not found errors when interacting with them.

Most of the removed authentication tasks don't appear because they live for a very short time but some still show up and exhibits this behavior.

Fixing this bug is in our roadmap

stevezemlicka

@julien-f
Awesome, thank you for the update. So if I'm hearing you correctly, this seems to be more of a minor annoyance and not necessarily anything harmful or detrimental. Thank you for verifying this is on the roadmap. I'll hang tight until it is patched. Thank you for all your work and efforts on this project!

julien-f

@stevezemlicka Exactly

fataugie

So I've got a similar issue, but with slightly different additional sticky logs. So in my pic, "A" refers to this topic, but I also have the list "B" that is persistent between reboots of the Xen instance and even the Host server.
"A" will disappear with a refresh of the page... "B" is like the walking dead....

Is there a location I can go to on the server and remove them via the cli?

julien-f

@fataugie Those successful authentication logs (B) may be related to an old bug.

You can indeed delete them with xo-cli, either one by one or all task logs:

# a specific task
xo-cli rest del tasks/$id

# all of them
xo-cli rest del tasks

fataugie

@julien-f Well, I tried to delete all.....and got this output:

tony@xenorchestra:~/XenOrchestraInstallerUpdater$ xo-cli rest del tasks
✖ TypeError [ERR_INVALID_URL]: Invalid URL
    at new NodeError (node:internal/errors:405:5)
    at new URL (node:internal/url:611:13)
    at Object.exec (file:///opt/xo/xo-builds/xen-orchestra-202312212125/packages/xo-cli/rest.mjs:152:21)
    at Object.del (file:///opt/xo/xo-builds/xen-orchestra-202312212125/packages/xo-cli/rest.mjs:46:33)
    at Object.rest (file:///opt/xo/xo-builds/xen-orchestra-202312212125/packages/xo-cli/rest.mjs:149:28)
    at async main (file:///opt/xo/xo-builds/xen-orchestra-202312212125/packages/xo-cli/index.mjs:396:14) {
  code: 'ERR_INVALID_URL',
  input: 'undefined'
}

olivierlambert

Have you registered your XO CLI to your XO instance first?

fataugie

@olivierlambert Well.....now I did.
After much hair pulling and gnashing of teeth, it worked as it should. I was unaware I needed to. Thanks

olivierlambert

It's documented here: https://xen-orchestra.com/docs/architecture.html#xo-cli-cli

Happy to know it works now

julien-f

@fataugie xo-cli will now display an explicit error in that case

https://github.com/vatesfr/xen-orchestra/commit/0d4cf48410840ee8e9dd783da0dec9d505ff47dd

0

julien-f committed to vatesfr/xen-orchestra

feat(xo-cli rest): explicit error if not registered

Fixes https://xcp-ng.org/forum/post/68698

fataugie

@julien-f Just one question.....do you guys EVER sleep?
I'm blown away at the responsiveness of this project....thanks for the efforts!

olivierlambert

And we are blown away by our wonderful community! Thank you very much, and happy to help you!