XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XCP-ng 7.5.0 final is here

    Scheduled Pinned Locked Moved Development
    74 Posts 15 Posters 32.8k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D Offline
      dsiminiuk @olivierlambert
      last edited by olivierlambert

      @olivierlambert I edited the properties of the VM and changed the "Home Server" property to server1 rather than do not assign. Then it was able to start. When I looked at XCP ng Center this morning the status icon for that VM was yellow. I restarted the tool stack on the server and the VM status turned green. I also received an email from the pool...

      Field                Value
      -----                -----
      Name:                PBD_PLUG_FAILED_ON_SERVER_START
      Priority:        3
      Class:                Host
      Object UUID:        c3970498-bc7f-4a49-8fe2-23e1e6dcb6bb
      Timestamp:        20180811T06:27:04Z
      Message UUID:        88db4c9a-6435-0bd3-2237-5d4d904a5782
      Pool name:        pool1
      

      Maybe related, maybe not. XCP ng Center crashed twice along the way and mentioned to look in the log in D:\Users\<my_user>\AppData\Roaming\XCP-ng\XCP-ng Center\logs\XCP-ng Center.log

      Recurring ERRORs

      2018-07-12 22:11:48,931 ERROR XenAdmin.Dialogs.LicenseStatus [23] - Couldn't get the server time for pool1 because: Object reference not set to an instance of an object.
      2018-07-12 22:11:48,931 ERROR XenAdmin.Utils.AsyncServerTime [23] - Failed to fetch server time for host pool1 because Object reference not set to an instance of an object.
      
      2018-07-12 22:34:27,141 ERROR XenAdmin.Program [Named pipe thread] - Exception in Invoke (ControlType=XenAdmin.MainWindow, MethodName=<ConnectPipe>b__1)
      System.ObjectDisposedException: Cannot access a disposed object.
      Object name: 'MainWindow'.
         at System.Windows.Forms.Control.MarshaledInvoke(Control caller, Delegate method, Object[] args, Boolean synchronous)
         at System.Windows.Forms.Control.Invoke(Delegate method, Object[] args)
         at XenAdmin.Program.Invoke(Control c, MethodInvoker f)
      
      • other error removed, too old to be pertinent to 7.5

      Everything appears stable at the moment but I'm hesitant to manipulate my VMs just for the sake of doing so. I've taken fresh backups of all my VMs so I have a restore point. If there are specific tests you would recommend to peel back the onion, I am willing to try.

      1 Reply Last reply Reply Quote 0
      • D Offline
        dsiminiuk
        last edited by

        I edited that VM again and removed the startup host preference. Shut it down. Started it up again and it worked. That's a good sign.

        1 Reply Last reply Reply Quote 0
        • D Offline
          dsiminiuk
          last edited by dsiminiuk

          More info. I made server2 the pool master and it started migrating all the VMs from server1.

          When it came to the only WIndows VM I have, I received an error on the migration dialog in XCP ng Center. From the log file...

          2018-08-12 14:45:42,302 INFO  Audit [21] - Operation success: DelegatedAsyncAction: pool1: Pool 86708278-0d7d-2bb1-5024-b04c39690e15 (pool1): Completed
          2018-08-12 14:45:42,393 WARN  XenAdmin.Network.TaskPoller [72] - Action failed due to API failure:
             at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
             at System.Environment.get_StackTrace()
             at XenAdmin.Network.TaskPoller.poll()
             at XenAdmin.Network.TaskPoller.PollToCompletion()
             at XenAdmin.Actions.EvacuateHostAction.Run()
             at XenAdmin.Actions.AsyncAction.RunWorkerThread(Object o)
             at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
             at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
             at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
             at System.Threading.ThreadPoolWorkQueue.Dispatch()
          2018-08-12 14:45:42,393 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'task.destroy'
          2018-08-12 14:45:42,434 ERROR XenAdmin.Actions.EvacuateHostAction [72] - There was an exception putting the host OpaqueRef:6d59f477-2e3c-40f1-920b-575719f33cee into maintenance mode.  Removing other_config key.
          2018-08-12 14:45:42,434 ERROR XenAdmin.Actions.EvacuateHostAction [72] - Internal error: Xenops_interface.Does_not_exist(_)
          Internal error: Xenops_interface.Does_not_exist(_)
          2018-08-12 14:45:42,435 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'host.remove_from_other_config'
          2018-08-12 14:45:42,475 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'Async.host.enable'
          2018-08-12 14:45:42,513 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'task.remove_from_other_config'
          2018-08-12 14:45:42,552 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'task.add_to_other_config'
          2018-08-12 14:45:42,592 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'task.remove_from_other_config'
          2018-08-12 14:45:42,633 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'task.add_to_other_config'
          2018-08-12 14:45:43,765 INFO  XenAPI.Session [72] - Invoking JSON-RPC method 'task.destroy'
          2018-08-12 14:45:43,806 ERROR XenAdmin.Actions.AsyncAction [72] - Internal error: Xenops_interface.Does_not_exist(_)
          2018-08-12 14:45:43,806 ERROR XenAdmin.Actions.AsyncAction [72] -    at XenAdmin.Network.TaskPoller.poll()
             at XenAdmin.Network.TaskPoller.PollToCompletion()
             at XenAdmin.Actions.EvacuateHostAction.Run()
             at XenAdmin.Actions.AsyncAction.RunWorkerThread(Object o)
          2018-08-12 14:45:43,808 WARN  Audit [72] - Operation failure: EvacuateHostAction: pool1: Pool 86708278-0d7d-2bb1-5024-b04c39690e15 (pool1): Host cb05539d-3359-4535-97a9-7f2b635f44fd (xs1): Server 'xs1' entering maintenance mode...
          

          That's the error I see on the dialog box that is doing the "Entering Maintenance mode" work of migrating the VMs.

          It happened on the same Windows VM as before and now it's shut down and the maint mode wizard allowed me to continue to migrate the last Linux VM. As soon as the pool master appeared to change I received another error popup box.
          There has been an unexpected error. Technical details are in the logfile saved here https://cloud.k5rwk.org/index.php/s/e9weBDcyJmYzQHK

          Bottom snip...

          2018-08-12 14:50:38,477 ERROR XenAdmin.InvokeHelper [Main program thread] - Exception in Invoke
          System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object.
             at XenAdmin.TabPages.GeneralTabPage.PoolAdditionalLicenseString()
             at XenAdmin.TabPages.GeneralTabPage.generateGeneralBox()
             at XenAdmin.TabPages.GeneralTabPage.BuildList()
             at System.Action.Invoke()
             at XenAdmin.OtherConfigAndTagsWatcher.OnTagsChanged()
             at XenAdmin.OtherConfigAndTagsWatcher.<>c.<connection_ConnectionStateChanged>b__25_0()
             --- End of inner exception stack trace ---
             at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
             at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
             at System.Delegate.DynamicInvokeImpl(Object[] args)
             at XenAdmin.InvokeHelper.<>c__DisplayClass4_0.<Invoke>b__0()
          2018-08-12 14:50:38,480 ERROR XenAdmin.Program [Main program thread] - Exception in Invoke
          System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object.
             at XenAdmin.TabPages.GeneralTabPage.PoolAdditionalLicenseString()
             at XenAdmin.TabPages.GeneralTabPage.generateGeneralBox()
             at XenAdmin.TabPages.GeneralTabPage.BuildList()
             at System.Action.Invoke()
             at XenAdmin.OtherConfigAndTagsWatcher.OnTagsChanged()
             at XenAdmin.OtherConfigAndTagsWatcher.<>c.<connection_ConnectionStateChanged>b__25_0()
             --- End of inner exception stack trace ---
             at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
             at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
             at System.Delegate.DynamicInvokeImpl(Object[] args)
             at XenAdmin.InvokeHelper.<>c__DisplayClass4_0.<Invoke>b__0()
             --- End of inner exception stack trace ---
             at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
             at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
             at System.Delegate.DynamicInvokeImpl(Object[] args)
             at XenAdmin.Program.<>c__DisplayClass66_0.<Invoke>b__0()
          2018-08-12 14:50:38,481 FATAL XenAdmin.Program [Connection to 192.168.1.15] - Uncaught exception
          System.NullReferenceException: Object reference not set to an instance of an object.
             at System.Windows.Forms.Control.MarshaledInvoke(Control caller, Delegate method, Object[] args, Boolean synchronous)
             at System.Windows.Forms.Control.Invoke(Delegate method, Object[] args)
             at XenAdmin.Program.Invoke(Control c, Delegate f, Object[] p)
             at XenAdmin.MainWindow.System.ComponentModel.ISynchronizeInvoke.Invoke(Delegate method, Object[] args)
             at XenAdmin.OtherConfigAndTagsWatcher.connection_ConnectionStateChanged(Object sender, EventArgs e)
             at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
             at XenAdmin.Network.XenConnection.OnConnectionStateChanged()
             at XenAdmin.Network.XenConnection.EndConnect(Boolean clearCache, ConnectTask task, Boolean exiting)
             at XenAdmin.Network.XenConnection.HandleConnectionLost(ConnectTask task, Pool pool, String poolopaqueref)
             at XenAdmin.Network.XenConnection.HandleConnectionResult(ConnectTask task, Exception error, Pool pool)
             at XenAdmin.Network.XenConnection.ConnectWorkerThread(Object o)
             at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
             at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
             at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
             at System.Threading.ThreadHelper.ThreadStart(Object obj)
          
          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by olivierlambert

            @dsiminiuk Please use the Markdown syntax for code/text block

            ```
            your code
            ```
            

            edit: I edited your post but please use it next time.

            1 Reply Last reply Reply Quote 0
            • D Offline
              dsiminiuk
              last edited by

              Thanks! will do.

              1 Reply Last reply Reply Quote 0
              • D Offline
                dsiminiuk
                last edited by dsiminiuk

                I get the Internal error: Xenops_interface.Does_not_exist(_) when I migrate the Windows VM.
                I have to restart the toolostack to allow starting of the same VM on the target host in the pool.

                1 Reply Last reply Reply Quote 0
                • borzelB Offline
                  borzel XCP-ng Center Team
                  last edited by

                  What's about powering off the VM, moving, powering on?

                  D 2 Replies Last reply Reply Quote 0
                  • D Offline
                    dsiminiuk @borzel
                    last edited by

                    @borzel I'll try that. Thanks.

                    My most recent test was to have all the VMs running on server1, put it into Maint mode and watch all the VMs mmigrate to server2. It behaves the same way with the same error regardless of direction.

                    1 Reply Last reply Reply Quote 0
                    • D Offline
                      dsiminiuk @borzel
                      last edited by dsiminiuk

                      @borzel If I power off the VM gracefully I can start it on either server in the pool. I have to restart the toolstack on the target server following the error during a live migration and then I can start it on either machine from what appears to be a forcibly power off state (Start windows normally?).

                      It only happens during a live migration and so far only on this VM which happens to be Windows 2008 R2 and guest tools v7.1.

                      borzelB 1 Reply Last reply Reply Quote 0
                      • borzelB Offline
                        borzel XCP-ng Center Team @dsiminiuk
                        last edited by borzel

                        @dsiminiuk can you please create a full copy of this VM and livemigrade with that? Just to exclude some crude problems with the VDI's

                        D 4 Replies Last reply Reply Quote 0
                        • D Offline
                          dsiminiuk @borzel
                          last edited by

                          @borzel Running full clone copy to new storage... (not fast clone). Standby...

                          1 Reply Last reply Reply Quote 0
                          • D Offline
                            dsiminiuk @borzel
                            last edited by

                            @borzel The VM was cloned successfully and booted up fine.
                            I tried to live migrate it from the pool master (currently server1) to server2 and it's been "Migrating" for 20 minutes now, with no sign of progress. Waiting for it to succeed or fail either way. I deleted the logs before starting the migration so I'll have smaller files.

                            1 Reply Last reply Reply Quote 0
                            • D Offline
                              dsiminiuk @borzel
                              last edited by dsiminiuk

                              @borzel After 36 minutes I restarted the toolstack on the pool master.
                              The GUI gave an exception.
                              From the log...

                              2018-08-12 21:22:35,248 FATAL XenAdmin.Program [Connection to xs1] - Uncaught exception
                              System.NullReferenceException: Object reference not set to an instance of an object.
                              

                              Closed it, opened it. The Copied VM was still running on the pool master. I'm assuming the migration task was killed so that makes sense.
                              Now that the toolstack was restarted I'm trying live migrate it again.

                              I'm going to leave it overnight now and I'll report the outcome tomorrow (Monday) morning.

                              1 Reply Last reply Reply Quote 0
                              • txsastreT Offline
                                txsastre @olivierlambert
                                last edited by txsastre

                                @olivierlambert said in XCP-ng 7.5.0 final is here:

                                xensource

                                here you are !

                                may be you're gonna receive a warning, but its a save server, still havent installed an SSL certificate 😕

                                /var/log/xensource.log

                                https://own.incaciutat.com:10001/nextcloud/index.php/s/Mc9z7EzFtTe9KGB

                                /var/log/SMlog

                                https://own.incaciutat.com:10001/nextcloud/index.php/s/K72EwMkmxxGGeMW

                                1 Reply Last reply Reply Quote 0
                                • D Offline
                                  dsiminiuk @borzel
                                  last edited by dsiminiuk

                                  @borzel 12 hours 18 minutes later, the migration task still appears to be running with no progress indicated.

                                  I think restarting the toolstack will return it to "running" status and kill the migration task. I have not done so yet, awaiting your next advice.

                                  0_1534175808935_Image 6.png

                                  borzelB 1 Reply Last reply Reply Quote 0
                                  • borzelB Offline
                                    borzel XCP-ng Center Team @dsiminiuk
                                    last edited by

                                    @olivierlambert
                                    We ( @dsiminiuk and I ) did an remote session on his VM:

                                    • the PV-Tools from citrix where in a something strange state 😐
                                    • we tried to remove them and install the XCP-ng beta testsigned drivers, but without luck
                                      -> I will build me a test VM and play arround, to get a better understanding, what this Windows Server 2008 R2 is doing with our drivers
                                    1 Reply Last reply Reply Quote 1
                                    • borzelB Offline
                                      borzel XCP-ng Center Team
                                      last edited by borzel

                                      @dsiminiuk bad news, my fresh installed test VM (Server 2008 R2 with SP1) installed the (beta pre-alpha testsinged non-production do-not-use wear-gloves make-backups) drivers just fine:

                                      0_1534194874658_3b6ff56b-9658-4c4e-b596-1f849f878f23-grafik.png

                                      D 1 Reply Last reply Reply Quote 0
                                      • D Offline
                                        dsiminiuk @borzel
                                        last edited by

                                        I'm going to build a new Windows VM, anticipating the windows tools release. Thanks

                                        1 Reply Last reply Reply Quote 0
                                        • ? Offline
                                          A Former User @dsiminiuk
                                          last edited by A Former User

                                          @dsiminiuk
                                          I had a similar problem with my setup. 2 hosts with freenas-iscsi. Some VMs didn't migrate as they should, got stuck in a migrate/shutdown state 😃 I shutdown everything and start them again. That solved a few problems. One VM didn't find the hdd and when I tried restore from snapshot it didn't work 100% although it booted. But its just a "lab" and the VM was a pi-hole so just I re-installed it. After that I installed 1 one for redundancy 😉

                                          If you cant re-install it. Try shutdown it trough the cli.

                                          D 1 Reply Last reply Reply Quote 0
                                          • D Offline
                                            dsiminiuk @Guest
                                            last edited by dsiminiuk

                                            @fraggan @borzel
                                            I created a brand new Windows Server 2016 VM (named "sweet") on 50GB VDI.

                                            Edit: I then installed the guest tools extracted from the Xenserver ISO.

                                            Initiated live migration from server1 where I built it, to server2 (pool master)...

                                            2018-08-14 18:20:29,259 ERROR XenAdmin.Actions.AsyncAction [50] - Internal error: Xenops_interface.Does_not_exist(_)
                                            2018-08-14 18:20:29,259 ERROR XenAdmin.Actions.AsyncAction [50] -    at XenAdmin.Network.TaskPoller.poll()
                                               at XenAdmin.Network.TaskPoller.PollToCompletion()
                                               at XenAdmin.Actions.VMActions.VMMigrateAction.Run()
                                               at XenAdmin.Actions.AsyncAction.RunWorkerThread(Object o)
                                            2018-08-14 18:20:29,261 WARN  Audit [50] - Operation failure: VMMigrateAction: pool1: VM 2a80d18b-b1d6-d2c0-6874-e664b9ebdb67 (SWEET): Pool 86708278-0d7d-2bb1-5024-b04c39690e15 (pool1): Host c3970498-bc7f-4a49-8fe2-23e1e6dcb6bb (xs2): Migrating
                                            

                                            The new VM appears powered off attached to the pool.
                                            Initiated a power on of the VM on server2. Status stays Yellow, console of VM remains blank.
                                            I initiated a restart of the toolstack of server2 (pool master).
                                            The XCP-ng UI crashes with two popups to check the log.

                                            2018-08-14 18:55:54,296 ERROR XenAdmin.Program [Main program thread] - Exception in Invoke
                                            System.NullReferenceException: Object reference not set to an instance of an object.
                                             <snip>
                                            2018-08-14 18:55:54,297 ERROR XenAdmin.Program [19] - Exception in Invoke (ControlType=XenAdmin.MainWindow, MethodName=<RefreshTreeView>b__63_0)
                                            System.NullReferenceException: Object reference not set to an instance of an object.
                                               at System.Windows.Forms.Control.MarshaledInvoke(Control caller, Delegate method, Object[] args, Boolean synchronous)
                                             <snip>
                                            2018-08-14 18:55:54,304 FATAL XenAdmin.Program [19] - Uncaught exception
                                            

                                            After closing and reopening the UI, the VM I tried to start appeared powered off still.
                                            I initiated a startup of the new VM on server2 (pool master) and it started.
                                            It seems to me that for whatever reason the live migration fails, it leaves the target machine (server2 in this case) in an invalid state and you cannot start a VM on it again until the toolstack is restarted.

                                            D 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post