Power on function

dsiminiuk

Now that host1 is powered off.

Parameters seem ok.

power-on-mode ( RO): iLO
power-on-config (MRO): power_on_ip: 192.168.1.44; power_on_user: Administrator; power_on_password_secret: 94171423-9ac2-beac-6b87-81699a0a7de6

Run...

xe host-power-on host=cdb42b94-7ec6-47b4-aaef-d4c52f6bd7b3

The cursor sits at column 1 after the newline from the command and never times out.

I know this isn't an XO problem, but in case somebody else gets this far...

From the xensource.log

May 14 11:06:45 xcpng02 xapi: [ info||5026 UNIX /var/lib/xcp/xapi||cli] xe host-power-on host=cdb42b94-7ec6-47b4-aaef-d4c52f6bd7b3 username=root password=(omitted)
May 14 11:06:45 xcpng02 xapi: [ info||5026 UNIX /var/lib/xcp/xapi|session.login_with_password D:880acf326f2a|xapi] Session.create trackid=981ff975a6b71268bf97af6cf52b6c96 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
May 14 11:06:45 xcpng02 xapi: [debug||5027 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:f5013af2302b created by task D:880acf326f2a
May 14 11:06:45 xcpng02 xapi: [debug||5026 UNIX /var/lib/xcp/xapi|host.power_on R:97714a622dc3|audit] Host.power_on: host = 'cdb42b94-7ec6-47b4-aaef-d4c52f6bd7b3 (xcpng01)'
May 14 11:06:45 xcpng02 xapi: [debug||5029 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:aea00adbbc6a created by task D:5292ea4ccdb8
May 14 11:06:45 xcpng02 xapi: [ info||5029 UNIX /var/lib/xcp/xapi|session.logout D:d10858afe74a|xapi] Session.destroy trackid=220441f1f21a1564d410801d86460ab1
May 14 11:06:45 xcpng02 xapi: [debug||5030 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.slave_login D:2b34221718c5 created by task D:5292ea4ccdb8
May 14 11:06:45 xcpng02 xapi: [ info||5030 UNIX /var/lib/xcp/xapi|session.slave_login D:0072d08cd959|xapi] Session.create trackid=00f2456aa8d5ace26679a22fad62c265 pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49

I'm going to have to get into tcpdump to find out what is going on there.

olivierlambert

Are you trying to power on the master? If yes, it won't work, only the master can boot slaves.

dsiminiuk

@olivierlambert No, I'm on the master trying to power on the slave.

dsiminiuk

Async.host.power_on  (on xcpnghost1) 0%

Persists in the web UI Tasks panel no matter what I do.
Restarted toolstack,
Shut down the entire environment and restarted.
No way to cancel it or delete it.

olivierlambert

Are you sure it's not a XO refresh issue?

dsiminiuk

@olivierlambert It's been there since I played with power on function that failed.

Where is this task event stored? On the master? On XO?

How can I query it from a command line?

olivierlambert

xe task-list on any pool member

dsiminiuk

@olivierlambert said in Power on function:

xe task-list

Oh, that was too easy. There it is.

uuid ( RO)                : e1896255-fbec-76d6-c2f4-9c2829c1f72b
          name-label ( RO): Async.host.power_on
    name-description ( RO):
              status ( RO): pending
            progress ( RO): 0.000

xe task-cancel uuid=e1896255-fbec-76d6-c2f4-9c2829c1f72b

Doesn't cancel it.

olivierlambert

Okay and a toolstack restart on the master doesn't clean it?

dsiminiuk

@olivierlambert It did now but never did before

I wonder if it is because the same host the command was run against was the master at the time and is now. Perhaps server 2 as master previously failed when the command was on server 1.

Makes no sense to me. All good for now. Thank you.

dsiminiuk

@olivierlambert Nope, I didn't wait long enough. It's back.

uuid ( RO)                : 4d1971f4-4895-78c8-2856-d679d457b75e
          name-label ( RO): Async.host.power_on
    name-description ( RO):
              status ( RO): pending
            progress ( RO): 0.000

olivierlambert

It's not the same task (new UUID)

dsiminiuk

@olivierlambert correct, which tells me it has been added again by something else. XO? Where I initated the command originally?

olivierlambert

You can check who did this in the audit.log file of your master

dsiminiuk

@olivierlambert

Jul  8 10:58:09 xcpng01 xapi: [20200708T15:58:09.775Z|audit||104166 INET :::80|task.destroy D:b3c0abbbd938|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'OK' 'API' 'task.destroy' (('self' 'Async.host.power_on' '3450e4eb-9a11-24ba-5e0c-699ff352028b' 'OpaqueRef:99fd01c3-535d-4334-bb92-c4cb4f84f16c')))

Jul  8 11:51:16 xcpng01 xapi: [20200708T16:51:16.956Z|audit||108793 UNIX /var/lib/xcp/xapi|task.cancel D:05f4e18101a6|audit] ('trackid=e49c5c3a24fed047e0aa1a3ed7b47ab3' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'OK' 'API' 'task.cancel' (('task' 'Async.host.power_on' 'e1896255-fbec-76d6-c2f4-9c2829c1f72b' 'OpaqueRef:083c7658-c1e5-46d7-bf9c-3b809be635d6')))

olivierlambert

This is for task destroy, not a task creation.

dsiminiuk

@olivierlambert it is all I see from:

fgrep Async.host.power_on /var/log/audit.log

olivierlambert

Grep on host.power_on to see if it's better.

dsiminiuk

@olivierlambert

fgrep host.power_on audit.log.1 | fgrep -v destroy | fgrep -v cancel

Jul  7 23:58:02 xcpng01 xapi: [20200708T04:58:02.118Z|audit||36922 |Async.host.power_on R:0af8abcf2f46|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))

Jul  8 00:58:02 xcpng01 xapi: [20200708T05:58:02.803Z|audit||42462 |Async.host.power_on R:2733002e2e58|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))

Jul  8 01:58:03 xcpng01 xapi: [20200708T06:58:03.488Z|audit||48005 |Async.host.power_on R:e74dd3d29556|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))

Jul  8 02:58:04 xcpng01 xapi: [20200708T07:58:04.229Z|audit||53358 |Async.host.power_on R:e730dacdf348|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))

Jul  8 03:58:04 xcpng01 xapi: [20200708T08:58:04.897Z|audit||58846 |Async.host.power_on R:19f454d900b0|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))

Now going back in time to audit.log.[2-7].gz

olivierlambert

It's possible that XAPI recorder the host.power_on and loop on trying to do that, but to be fair, I never saw that before

Can you power on that host to see if it stops?