XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Can't start or shutdown VM's after storage was powered off

    Scheduled Pinned Locked Moved Compute
    5 Posts 2 Posters 2.2k Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D Offline
      dgleba
      last edited by dgleba

      This post will be less narrow and specific than we would all like - But it is my situation.

      I have :

      • Pool of 3 XCP-ng v8.0. on 3 Dell T5500's. Recently converted from ESXi - 3 months ago. Approx 50 VM's.
      • 2 separate FreeNAS storage computers.

      I have used XenServer 6 for years in a business environment.

      I Left HA and VM's running.
      I Shutdown one FreeNAS box due to disk error to investigate.

      Problem: (trio of servers)

      After turning FreeNAS back on, I could not do much with Xcp-ng.

      I can ssh.
      I can now use the XCP-ng center.
      Most VM's are starting, but not started.

      I tried to shut down VM's using command line xe commands per https://support.citrix.com/article/CTX220777
      I can not shut most of them down, I can't start them.
      I could not do much with the system.

      I used commands "like" these and others.

       96  xe pool-list
      101  /etc/init.d/xapissl restart
      104  service xapi status
      105  service xapi restart
      107  netstat -an | grep 172.16.2.30
      111  cat network.conf
      112  cat xapi-ssl.pem
      113  hostname -f
      116  xe host-list params=uuid,name-label,host-metrics-live
      117  xe host-is-in-emergency-mode
      121  xe host-emergency-ha-disable --force
      122  xe sr-list
      123  xe host-list params=uuid,name-label,host-metrics-live
      

      Test attempt:

      Not being able to use any particular XCP-ng server, I powered them all off. Then I set up a 4th fresh XCP-ng on another computer - Dell R620.

      Problem: (Dell R620)

      1. mounting exiting SR's takes much longer than before. Instead of 10 seconds, it's two minutes for example.
      2. The windows 2012 w/exchange 2013 VM takes hours to start and exchange services don't start. The network and sharing dialog won't load.

      I realize this sounds like a soap opera, but it is my situation. It probably sounds like a country song, I lost my job, then my car, then my wife, then my.....

      I will post the last error messages from the XCP-ng log later today.

      Many thanks for your patience with me.

      Update 12:50pm:

      I edited the post above in many places. Read it again if you already read it.
      There are no errors in the XCP-ng center log.

      Update 7:50pm:

      I clarified the post title and that the original three servers are powered off. The R620 is the only one running now.

      1 Reply Last reply Reply Quote 0
      • ForzaF Offline
        Forza
        last edited by

        Did you restart the entire pool? Sounds like you might have NFS issues like stale mounts lingering. This could make things very slow on the XCP-ng side. Do you export your NFS with fsid option? You should.

        1 Reply Last reply Reply Quote 0
        • D Offline
          dgleba
          last edited by

          @S-Pam said in Can't start or shutdown VM's after power outage:

          export your NFS with fsid option

          I am not certain what you mean by "export your NFS with fsid option"?
          Can you explain?

          1 Reply Last reply Reply Quote 0
          • D Offline
            dgleba
            last edited by

            Where would the stale mounts be? .. in XCP-ng? or in Freenas?

            1 Reply Last reply Reply Quote 0
            • ForzaF Offline
              Forza
              last edited by Forza

              In FreeNAS. exports file you should have fsid=123, with a unique number for each exported path. If not it can cause a problem if the nfs server is rebooted while the pool is up.

              https://linux.die.net/man/5/exports
              https://www.suse.com/support/kb/doc/?id=000017897

              The stale mounts are in xcp-ng.
              But this is just a theory.

              During the power outage. Did all xcp-ng hosts and the FreeNAS as boxes go down? How did you boot up? You wrote you set up fresh servers?

              1 Reply Last reply Reply Quote 0
              • First post
                Last post