xe command "hangs on start" when max / open files is high

[olivierlambert]

Hi!

Thanks for the feedback That's another good reason to move forward on our REST API in XO

[endi]

@olivierlambert

Looking forward to it: fyi, we are currently using this to facilitate the transfer of templates between clusters. We didn't find it possible to achieve that via xo-cli, though we're open to the possibility we failed to understand something

[olivierlambert]

Can you show us how do you achieve that with xe so we can check how/if it's possible to do it via XO?

[endi]

hi @olivierlambert

we have more than one "destination" server, but simplified here for only one:

check if destination server has a template with name "x" present
xe -s <server> -u <user> -pw <pw> template-list name-label=x

if present, do nothing.

if absent on this destination, check if we have that template locally
# this is just a filesystem check where we are running

if absent locally, pull it from our source server
# not captured here: lookup template uuid for 'x'
xe -s <server> -u <user> -pw <pw> template-export template-uuid='uuid' filename='templatename'

push to destination
xe -s <server> -u <user> -pw <pw> vm-import filename='templatename'

Where we have multiple targets, loop over remaining destinations, and push when not detected as being present.

So tldr:

template-list
template-export
vm-import

Are what I think we need. I wasn't the most recent to look at using xo-cli for this, but we either didn't find them all, or somehow didn't understand how to use what we need

cheers
ewan

[olivierlambert]

So all in all, the "functional" specification is to be sure to propagate templates between multiple pools, right? (and you are running this in a kind of CRON?)

[endi]

@olivierlambert

Yes, for reasons that would pollute this thread, we're running a number of single host pools.

The broader picture is:

We drive https://github.com/ddelnano/packer-plugin-xenserver from a CI server to build a small number of base images / templates

A related CI job then replicates those to all destination pools.

I don't think I'm yet in a position to share full details publicly, but if you're interested I'm sure I could give you more visibility to what we've built so far, and our intended direction of travel.

regards
ewan

[olivierlambert]

Clearly, having a central API point might be easier for that. Let me add a link to this topic internally for a next conversation with XO team

[marcungeschikts]

[olivierlambert]

@endi can you show us your current script so we can understand all the requirements?

[psafont]

There's ongoing effort to avoid opening file descriptors, to avoid this situation. I'll consult whether the current patches avoid this situation in template-export / vm-import

https://github.com/xapi-project/xen-api/pull/4877

snwoods opened this pull request in xapi-project/xen-api

closed CP-32622: avoid using select and instead use epoll #4877