XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Broken Host Consoles in fresh xcp-ng 8 installation. VM consoles work.

    Scheduled Pinned Locked Moved Solved Xen Orchestra
    displayconsolegpu
    27 Posts 6 Posters 10.8k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R Offline
      rjt @rjt
      last edited by rjt

      Example from another machine on which the IP address of the Control domain VM does not match the actual IP address of Dom0 192.168.2.141 != 192.168.10.192:

      [01:23 eceoxen-B ~]# xe console-list vm-name-label=Control\ domain\ on\ host:\ eceoxen-B
      uuid ( RO)             : d5039d1a-64ad-c8a9-a309-51e568ba2926
                vm-uuid ( RO): 1593da28-8e85-4252-878e-778eb414c549
          vm-name-label ( RO): Control domain on host: eceoxen-B
               protocol ( RO): VT100
               location ( RO): https://192.168.2.141/console?ref=OpaqueRef:7be93ca1-76cf-4649-8276-74891eac0a06
      
      uuid ( RO)             : 7bcf725e-ae83-cf6f-7997-7dd63469929f
                vm-uuid ( RO): 1593da28-8e85-4252-878e-778eb414c549
          vm-name-label ( RO): Control domain on host: eceoxen-B
               protocol ( RO): RFB
               location ( RO): https://192.168.2.141/console?ref=OpaqueRef:1361b935-2d25-4929-9b27-4b0483cbb0f7
      
      [01:25 eceoxen-B ~]# ip a show  | egrep inet | egrep -v '(inet 127)'
          inet 192.168.10.192/16 brd 192.168.255.255 scope global dynamic xenbr0
      
      1 Reply Last reply Reply Quote 0
      • R Offline
        rjt
        last edited by rjt

        Attempts to reset or clear the Dom0 console entries ....

        [02:37 xcp-ng-G ~]# xe console-param-clear param-name=location uuid=fb69931e-ae2b-c2d3-dbce-3e9a9ae57646
        Error: Can only clear RW parameters
        
        [02:38 xcp-ng-G ~]# xe console-param-clear uuid=12aae74a-4d4f-7ed3-976d-b36e0ae1905d param-name=location
        Error: Can only clear RW parameters
        
        [02:38 xcp-ng-G ~]# xe console-param-remove uuid=12aae74a-4d4f-7ed3-976d-b36e0ae1905d param-key=location param-name=location
        Error: Can only remove from parameters of type Set or Map
        
        [02:38 xcp-ng-G ~]# xe console-param-set uuid=12aae74a-4d4f-7ed3-976d-b36e0ae1905d
        

        Is there some way to blow these old Dom0 consoles away?

        1 Reply Last reply Reply Quote 0
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          Does a reboot fix anything? Try to see if you have network parameters set somewhere. Maybe XCP-ng Center saved some config?

          1 Reply Last reply Reply Quote 0
          • R Offline
            rjt
            last edited by rjt

            I have rebooted the hosts numerous times. xcp center configuration looks good.
            Tried reapplying network config, but nothing.
            Tried the xsconsole emergency network reset.
            Tried the firstboot service and /etc/firstboot.d/, but no luck.
            Wonder if it is picking up configuration information from another harddrive used to boot xcp.
            Resigned to wipe and reinstall.

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              I don't know how this thing might be stuck in your config. At worst you can edit XAPI DB manually and change the value.

              1. Stop XAPI service on all hosts of the pool
              2. Copy /var/xapi/state.db somewhere else in case
              3. Edit it, and find the bad IP, replace it with the right one
              4. Save
              5. Start XAPI on all hosts, master first
              R 1 Reply Last reply Reply Quote 0
              • R Offline
                rjt @olivierlambert
                last edited by

                @olivierlambert
                The /var/lib/xcp/state.db is basically a single line of xml with a half-million characters.
                The old egrep or sed or vi searching did not seem to narrow things down likely they normally do. Used xmllint to put the db in a pretty format with line breaks and indentation. Then vi to edit. Of course, as soon as i start xapi, those changes are lost. I assume there must be a massive speedup in lookups when all on a single line.

                [23:59 xen-B xcp]# pushd /var/lib/xcp/
                [23:59 xen-B xcp]# systemctl stop xapi
                [23:59 xen-B xcp]# cp state.db ./state-YYYYMMDD-HHMM.db
                [23:59 xen-B xcp]# egrep  '(192.168.2.141)' ./state.db | wc
                      1   11757  454034
                [23:59 xen-B xcp]# echo " :( 1 line with over 11,000 words :(" 
                [23:59 xen-B xcp]# xmllint --format state.db >> state.xmllint--pretty.db
                [23:59 xen-B xcp]# egrep  '(192.168.2.141)' ./state.db | wc
                     10     193   13526
                [23:59 xen-B xcp]# echo "i can deal with 193 words :) and edited with vi" 
                [23:59 xen-B xcp]# mv state.xmllint--pretty.db ./state.db
                [23:59 xen-B xcp]# systemctl start xapi
                

                xe console-list indicates the the proper IP addresses:

                [23:59 xen-B xcp]# xe console-list  vm-name-label=Control\ domain\ on\ host:\ xen-B
                
                uuid ( RO)             : d5039d1a-64ad-c8a9-a309-51e568ba2926
                          vm-uuid ( RO): 1593da28-8e85-4252-878e-778eb414c549
                    vm-name-label ( RO): Control domain on host: xen-B
                         protocol ( RO): VT100
                         location ( RO): https://192.168.10.192/console?ref=OpaqueRef:7be93ca1-76cf-4649-8276-74891eac0a06
                
                
                uuid ( RO)             : 7bcf725e-ae83-cf6f-7997-7dd63469929f
                          vm-uuid ( RO): 1593da28-8e85-4252-878e-778eb414c549
                    vm-name-label ( RO): Control domain on host: xen-B
                         protocol ( RO): RFB
                         location ( RO): https://192.168.10.192/console?ref=OpaqueRef:1361b935-2d25-4929-9b27-4b0483cbb0f7
                
                
                [00:00 xen-B xcp]# ip a show dev xenbr0 | egrep inet
                    inet 192.168.10.192/16 brd 192.168.255.255 scope global dynamic xenbr0
                

                But there is still a blank host console in both XOA and xcp center. Suppose those OpaqueRefs have to be fixed up as well. EFI starting crashing on this r720, so having many other severe issues to deal with.

                1 Reply Last reply Reply Quote 0
                • R Offline
                  rjt
                  last edited by

                  Anybody have an idea of what to do to get our consoles back?

                  1 Reply Last reply Reply Quote 0
                  • borzelB Offline
                    borzel XCP-ng Center Team
                    last edited by borzel

                    Do a recursive grep in /etc to find all files that have the old IP address

                    Like grep -R 'IP-Address' /etc

                    R 1 Reply Last reply Reply Quote 0
                    • R Offline
                      rjt @borzel
                      last edited by rjt

                      @borzel i had tried that but found grep did not work like i am used. egrep worked better, but still not like how i am used to..

                       egrep -R '(10\.40\.|192\.168\.)'  /etc
                      
                      1 Reply Last reply Reply Quote 0
                      • D Offline
                        DreDay
                        last edited by

                        I've seen that before where when you initially set the management IP via DHCP it breaks the console when the IP changes. XCP-ng does seem to register that the IP has changed and so it does not update the console URI.

                        We were able to resolve the issue by setting the IP statically on the XCP-ng node(I believe a reboot was also required). That updated the console's URI and it was working again in XCP-ng Center and XO.

                        R 1 Reply Last reply Reply Quote 1
                        • R Offline
                          rjt @DreDay
                          last edited by

                          @DreDay yes, statically leased DHCP addresses that used to work on spinning platters of rust no longer work on Solid State Drives.

                          However, setting static IP address did not work to fix it. Had to reinstall, setting a static IP at re-installation time.

                          1 Reply Last reply Reply Quote 0
                          • A Offline
                            asdfonseca @olivierlambert
                            last edited by

                            @olivierlambert , i have the same problem, but only under NAT. In same subnet, both host and VM consoles work properly.

                            There's any special configuration to work under NAT?

                            1 Reply Last reply Reply Quote 0
                            • olivierlambertO Offline
                              olivierlambert Vates 🪐 Co-Founder CEO
                              last edited by

                              When XO will ask to access host console, XCP-ng XAPI will return the local host console IP (in the same network).

                              So if you XO can't reach this IP, console connection can't work.

                              A 2 Replies Last reply Reply Quote 1
                              • A Offline
                                asdfonseca @olivierlambert
                                last edited by

                                @olivierlambert, all right! Thanks for your explanation!

                                1 Reply Last reply Reply Quote 0
                                • A Offline
                                  asdfonseca @olivierlambert
                                  last edited by

                                  @olivierlambert, just one more question for the XCP-Ng development team:

                                  Would it be possible for XCP-ng XAPI to read and return, for example, the addresses contained in the [/etc/hosts] file? In this case, if we put both the local IP and the external address used to access NAT on /etc/hosts and do a careful reading and interpretation of its content, it would act analogously to a DNS to translate both the local address and the external to XAPI.

                                  Could it be that when implementing a change like this in XCP-Ng, would XAPI work under NAT?

                                  1 Reply Last reply Reply Quote 0
                                  • olivierlambertO Offline
                                    olivierlambert Vates 🪐 Co-Founder CEO
                                    last edited by

                                    Short answer: no.
                                    Long answer: I'm not sure on the "how" we could do that. You should probably ask a XAPI developer about this. I'll try to get one or two here 🙂

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post