Power on function



  • @olivierlambert correct, which tells me it has been added again by something else. XO? Where I initated the command originally?


  • XCP-ng Team

    You can check who did this in the audit.log file of your master 🙂



  • @olivierlambert

    Jul  8 10:58:09 xcpng01 xapi: [20200708T15:58:09.775Z|audit||104166 INET :::80|task.destroy D:b3c0abbbd938|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'OK' 'API' 'task.destroy' (('self' 'Async.host.power_on' '3450e4eb-9a11-24ba-5e0c-699ff352028b' 'OpaqueRef:99fd01c3-535d-4334-bb92-c4cb4f84f16c')))
    
    Jul  8 11:51:16 xcpng01 xapi: [20200708T16:51:16.956Z|audit||108793 UNIX /var/lib/xcp/xapi|task.cancel D:05f4e18101a6|audit] ('trackid=e49c5c3a24fed047e0aa1a3ed7b47ab3' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'OK' 'API' 'task.cancel' (('task' 'Async.host.power_on' 'e1896255-fbec-76d6-c2f4-9c2829c1f72b' 'OpaqueRef:083c7658-c1e5-46d7-bf9c-3b809be635d6')))
    

  • XCP-ng Team

    This is for task destroy, not a task creation.



  • @olivierlambert it is all I see from:

    fgrep Async.host.power_on /var/log/audit.log


  • XCP-ng Team

    Grep on host.power_on to see if it's better.



  • @olivierlambert

    fgrep host.power_on audit.log.1 | fgrep -v destroy | fgrep -v cancel

    Jul  7 23:58:02 xcpng01 xapi: [20200708T04:58:02.118Z|audit||36922 |Async.host.power_on R:0af8abcf2f46|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))
    
    Jul  8 00:58:02 xcpng01 xapi: [20200708T05:58:02.803Z|audit||42462 |Async.host.power_on R:2733002e2e58|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))
    
    Jul  8 01:58:03 xcpng01 xapi: [20200708T06:58:03.488Z|audit||48005 |Async.host.power_on R:e74dd3d29556|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))
    
    Jul  8 02:58:04 xcpng01 xapi: [20200708T07:58:04.229Z|audit||53358 |Async.host.power_on R:e730dacdf348|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))
    
    Jul  8 03:58:04 xcpng01 xapi: [20200708T08:58:04.897Z|audit||58846 |Async.host.power_on R:19f454d900b0|audit] ('trackid=68849bbe6e0a22d9b0b024c21c2b6e36' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'ERROR:INTERNAL_ERROR: [ (Failure \"The host failed to power on.\") ]' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))
    

    Now going back in time to audit.log.[2-7].gz


  • XCP-ng Team

    It's possible that XAPI recorder the host.power_on and loop on trying to do that, but to be fair, I never saw that before 😄

    Can you power on that host to see if it stops?



  • @olivierlambert

    Going back as far as I can to audit.log.7.gz only takes me back to June 30 and this has been a problem for more than a month so I don't think I can find the original request.



  • @olivierlambert said in Power on function:

    Can you power on that host to see if it stops?

    I had the whole rack powered off yesterday, 2 - XCP-NG 8.1 servers (DL380 Gen9), 1 Freenas (DL380 Gen9) for iSCSI VDIs. It still came back.


  • XCP-ng Team

    So now everything is back online but despite that you got the task?



  • @olivierlambert said in Power on function:

    So now everything is back online but despite that you got the task?

    Yes, everything is up and working, just the task appears in XO and also on the master's tasks list as we see.


  • XCP-ng Team

    Weird. Can you disconnect XO from this pool, restart toolstack on master, and see if the task is popping again? I think it will, I don't see why XO would do that.



  • @olivierlambert said in Power on function:

    Weird. Can you disconnect XO from this pool, restart toolstack on master, and see if the task is popping again? I think it will, I don't see why XO would do that.

    I would remove the hosts from XO, restart toolstack, add hosts back to XO? Is this correct?


  • XCP-ng Team

    Don't add host back in XO for few minutes, and check if xe task-list got the task appearing without any XO connection.



  • @olivierlambert

    Host #2 has been powered off during these checks. I will power it on so that all pool members are available and try xe task-cancel again. Then proceed with your suggestion. Standby.



  • @olivierlambert

    OK this is interesting (host1 is master). I powered on host2 and ... the task disappeared from xe task-list and XO Tasks.

    I powered off host2 again and the task is back.

    [12:38 xcpng01 log]# xe task-list params=all
    uuid ( RO)                  : 221a86ff-ae3f-d10e-26bc-9a7b693f44f0
                name-label ( RO): Async.host.power_on
          name-description ( RO):
                subtask_of ( RO): <not in database>
                  subtasks ( RO):
               resident-on ( RO): cdb42b94-7ec6-47b4-aaef-d4c52f6bd7b3
                    status ( RO): pending
                  progress ( RO): 0.000
                      type ( RO): <none/>
                    result ( RO):
                   created ( RO): 20200708T17:37:00Z
                  finished ( RO): 19700101T00:00:00Z
                error_info ( RO):
        allowed_operations ( RO): Cancel
        current_operations ( RO):
              other-config (MRW):
    


  • I am powering up host2 again only to remove it from XO and follow your direction.



  • @olivierlambert

    Both servers are running in the pool.
    Both removed from XO.
    Toolstack restarted on both servers via XCP-NG Center at 12:50pm local time.
    Server communications re-established automatically in XCP-NG Center at 12:52pm.
    Pool master added and 2nd server added to XO at 12:54pm.
    No tasks are listed on either server with xe task-list.
    No tasks are listed in the XO UI.
    Powered off server 2 from the server's console screen (not XCP-NG Center or XO being the point).

    I have a task in the master on the CLI and XO again.

    uuid ( RO)                : 2f820b63-f03a-1258-1443-2f946569a4fb
              name-label ( RO): Async.host.power_on
        name-description ( RO):
                  status ( RO): pending
                progress ( RO): 0.000
    
    

    The only mention of power_on in audit.log since the power up time...

    Jul  8 12:48:15 xcpng01 xapi: [20200708T17:48:15.096Z|audit||3666 |Async.host.power_on R:9802e63aaa34|audit] ('trackid=5835f391b350762fbb9dc4eb46b25614' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'OK' 'API' 'host.power_on' (('host' 'xcpng02' '1139bfbc-c487-4311-8206-bbffe96ed6b1' 'OpaqueRef:426e950f-f926-490f-b598-328314aafbdc')))
    
    Jul  8 12:48:15 xcpng01 xapi: [20200708T17:48:15.183Z|audit||4625 INET :::80|task.destroy D:39825944bdf5|audit] ('trackid=5835f391b350762fbb9dc4eb46b25614' 'LOCAL_SUPERUSER' 'root' 'ALLOWED' 'OK' 'API' 'task.destroy' (('self' 'Async.host.power_on' '221a86ff-ae3f-d10e-26bc-9a7b693f44f0' 'OpaqueRef:9802e63a-aa34-4f8e-9876-94f0b3778324')))
    


  • @dsiminiuk You should only be adding the pool master in XO, not the slaves.


Log in to reply
 

XCP-ng Pro Support

XCP-ng Pro Support