XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XOSTOR hyperconvergence preview

    Scheduled Pinned Locked Moved XOSTOR
    457 Posts 50 Posters 531.2k Views 53 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • BlueToastB Offline
      BlueToast
      last edited by BlueToast

      Last week I stood up two XCP-ng hosts with LINSTOR without issue.

      This week I am standing up a third XCP-ng host and encounter a show stopping error when I try to install LINSTOR:

      Error: Package: xcp-ng-linstor-1.1-3.xcpng8.2.noarch (xcp-ng-linstor)
      Requires: sm-linstor
      ...
      Failed to install LINSTOR package: xcp-ng-linstor.

      This third host is 100% identical in hardware to the first two hosts. Fresh from-scratch new install of XCP-ng. Here is an output of commands and their results on the third host:

      [21:35 bol-xcp3 ~]# wget https://gist.githubusercontent.com/Wescoeur/7bb568c0e09e796710b0ea966882fcac/raw/052b3dfff9c06b1765e51d8de72c90f2f90f475b/gistfile1.txt -O install && chmod +x install
      --2024-11-16 21:35:08--  https://gist.githubusercontent.com/Wescoeur/7bb568c0e09e796710b0ea966882fcac/raw/052b3dfff9c06b1765e51d8de72c90f2f90f475b/gistfile1.txt
      Resolving gist.githubusercontent.com (gist.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...
      Connecting to gist.githubusercontent.com (gist.githubusercontent.com)|185.199.111.133|:443... connected.
      HTTP request sent, awaiting response... 200 OK
      Length: 3596 (3.5K) [text/plain]
      Saving to: โ€˜installโ€™
      
      100%[=============================================================================>] 3,596       --.-K/s   in 0s      
      
      2024-11-16 21:35:08 (22.4 MB/s) - โ€˜installโ€™ saved [3596/3596]
      
      [21:35 bol-xcp3 ~]# ./install --disks /dev/nvme0n1 --thin
      Loaded plugins: fastestmirror
      Loading mirror speeds from cached hostfile
      Excluding mirror: updates.xcp-ng.org
       * xcp-ng-base: mirrors.xcp-ng.org
      Excluding mirror: updates.xcp-ng.org
       * xcp-ng-updates: mirrors.xcp-ng.org
      Resolving Dependencies
      --> Running transaction check
      ---> Package xcp-ng-release-linstor.noarch 0:1.3-1.xcpng8.2 will be installed
      --> Finished Dependency Resolution
      
      Dependencies Resolved
      
      =======================================================================================================================
       Package                             Arch                Version                     Repository                   Size
      =======================================================================================================================
      Installing:
       xcp-ng-release-linstor              noarch              1.3-1.xcpng8.2              xcp-ng-updates              4.0 k
      
      Transaction Summary
      =======================================================================================================================
      Install  1 Package
      
      Total download size: 4.0 k
      Installed size: 477  
      Downloading packages:
      xcp-ng-release-linstor-1.3-1.xcpng8.2.noarch.rpm                                                | 4.0 kB  00:00:01     
      Running transaction check
      Running transaction test
      Transaction test succeeded
      Running transaction
        Installing : xcp-ng-release-linstor-1.3-1.xcpng8.2.noarch                                                        1/1 
        Verifying  : xcp-ng-release-linstor-1.3-1.xcpng8.2.noarch                                                        1/1 
      
      Installed:
        xcp-ng-release-linstor.noarch 0:1.3-1.xcpng8.2                                                                       
      
      Complete!
      Loaded plugins: fastestmirror
      Loading mirror speeds from cached hostfile
      Excluding mirror: updates.xcp-ng.org
       * xcp-ng-base: mirrors.xcp-ng.org
      Excluding mirror: updates.xcp-ng.org
       * xcp-ng-updates: mirrors.xcp-ng.org
      Resolving Dependencies
      --> Running transaction check
      ---> Package xcp-ng-linstor.noarch 0:1.1-3.xcpng8.2 will be installed
      --> Processing Dependency: sm-linstor for package: xcp-ng-linstor-1.1-3.xcpng8.2.noarch
      --> Finished Dependency Resolution
      Error: Package: xcp-ng-linstor-1.1-3.xcpng8.2.noarch (xcp-ng-linstor)
                 Requires: sm-linstor
                 Available: sm-2.30.8-2.1.0.linstor.4.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-2.1.0.linstor.5.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-2.1.0.linstor.6.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-2.3.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-2.3.0.linstor.2.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-7.1.0.linstor.2.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-10.1.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-10.1.0.linstor.2.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-10.1.0.linstor.3.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-12.1.0.linstor.2.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-12.1.0.linstor.3.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Available: sm-2.30.8-12.1.0.linstor.4.xcpng8.2.x86_64 (xcp-ng-linstor)
                     sm-linstor
                 Installed: sm-2.30.8-13.1.xcpng8.2.x86_64 (@xcp-ng-updates)
                     Not found
                 Available: sm-2.29.1-1.2.xcpng8.2.x86_64 (xcp-ng-base)
                     Not found
                 Available: sm-2.30.4-1.1.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.4-1.1.0.linstor.8.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.6-1.1.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.6-1.1.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.6-1.2.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.1.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.2.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.3.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.7-1.3.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.3.0.linstor.2.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.3.0.linstor.3.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.3.0.linstor.7.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.7-1.3.0.linstor.8.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.8-2.1.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.8-2.1.0.linstor.1.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.8-2.1.0.linstor.2.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.8-2.1.0.linstor.3.xcpng8.2.x86_64 (xcp-ng-linstor)
                     Not found
                 Available: sm-2.30.8-2.3.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.8-7.1.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.8-10.1.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
                 Available: sm-2.30.8-12.1.xcpng8.2.x86_64 (xcp-ng-updates)
                     Not found
       You could try using --skip-broken to work around the problem
       You could try running: rpm -Va --nofiles --nodigest
      Failed to install LINSTOR package: xcp-ng-linstor.
      

      It seems the sm* package(s) are missing ? I don't know what to do.

      Edit 2024-11-17 1702 CST: yum install xcp-ng-release-linstor indicates xcp-ng-release-linstor is already installed and latest version.

      [16:14 bol-xcp3 ~]# yum install xcp-ng-release-linstor
      Loaded plugins: fastestmirror
      Loading mirror speeds from cached hostfile
      Excluding mirror: updates.xcp-ng.org
       * xcp-ng-base: mirrors.xcp-ng.org
      Excluding mirror: updates.xcp-ng.org
       * xcp-ng-updates: mirrors.xcp-ng.org
      Package xcp-ng-release-linstor-1.3-1.xcpng8.2.noarch already installed and latest version
      Nothing to do
      
      DanpD 1 Reply Last reply Reply Quote 0
      • DanpD Offline
        Danp Pro Support Team @BlueToast
        last edited by

        @BlueToast This should be fixed now. Please retry the XOSTOR installation.

        BlueToastB 1 Reply Last reply Reply Quote 1
        • BlueToastB Offline
          BlueToast @Danp
          last edited by

          @Danp Success this this - thanks for the assist. ๐Ÿ™‚ Executed with great success:

          yum install xcp-ng-linstor
          yum install xcp-ng-release-linstor
          ./install --disks /dev/nvme0n1 --thin
          
          1 Reply Last reply Reply Quote 0
          • G Offline
            gb.123 @olivierlambert
            last edited by

            @olivierlambert

            Since XOSTOR is now supported on XCP-ng 8.3 LTS, should we use the same script or some other method is required ?
            Can you remove the heading which states the script is only compatible with 8.2 ?

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates ๐Ÿช Co-Founder CEO
              last edited by

              Ping @Team-Storage

              1 Reply Last reply Reply Quote 0
              • P Offline
                peter_webbird
                last edited by

                Is CBT meant to be supported on XOSTOR?

                I've been experimenting with XOSTOR recently, but upon testing a delta-backup, noticed this warning...
                couldn't enable CBT

                The error message behind this is SR_OPERATION_NOT_SUPPORTED when calling Async.VDI.enable_cbt.

                Running xe sr-param-list uuid={uuid} shows the following:

                [~]# xe sr-param-list uuid={...}
                    uuid ( RO) : {...}
                    name-label ( RW): CD6
                    name-description ( RW): Array of Kioxia CD6 U.2 drives, one in each Host.
                    host ( RO): <shared>
                    allowed-operations (SRO): unplug; plug; PBD.create; update; PBD.destroy; VDI.resize; VDI.clone; scan; VDI.snapshot; VDI.mirror; VDI.create; VDI.destroy
                    {...etc}
                    type ( RO): linstor
                    content-type ( RO): user
                    shared ( RW): true
                    introduced-by ( RO): <not in database>
                    is-tools-sr ( RO): false
                    other-config (MRW): auto-scan: true
                    sm-config (MRO):
                    {...etc}
                

                Compared to another SR, the following allowed-operations are missing:

                VDI.enable_cbt;
                VDI.list_changed_blocks;
                VDI.disable_cbt;
                VDI.data_destroy;
                VDI.set_on_boot
                

                Is this the expected behaviour? Note that this is using XCP-ng 8.2 (I've yet to test out 8.3).

                ronan-aR 1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates ๐Ÿช Co-Founder CEO
                  last edited by

                  Ping @ronan-a or @dthenot

                  1 Reply Last reply Reply Quote 0
                  • ronan-aR Offline
                    ronan-a Vates ๐Ÿช XCP-ng Team @peter_webbird
                    last edited by ronan-a

                    @peter_webbird We've already had feedback on CBT and LINSTOR/DRBD, we don't necessarily recommend enabling it. We have a blocking dev card regarding a bug with LVM lvchange command that may fail on CBT volumes used by a XOSTOR SR. We also have other issues related to migration with CBT.

                    1 Reply Last reply Reply Quote 2
                    • G Offline
                      gb.123
                      last edited by

                      @ronan-a @dthenot @Team-Storage

                      Guys, Can you please clarify which method to use for installing XOSTOR in XCP-ng 8.3 ?

                      Simple :

                      yum install xcp-ng-linstor
                      yum install xcp-ng-release-linstor
                      ./install --disks /dev/nvme0n1 --thin
                      

                      Or the script in the first post ?
                      Or Some other script ?

                      dthenotD 1 Reply Last reply Reply Quote 2
                      • dthenotD Offline
                        dthenot Vates ๐Ÿช XCP-ng Team @gb.123
                        last edited by

                        @gb.123 Hello,
                        The instruction in the first post are still the way to go ๐Ÿ™‚

                        J 1 Reply Last reply Reply Quote 3
                        • J Offline
                          JeffBerntsen Top contributor @dthenot
                          last edited by

                          @dthenot said in XOSTOR hyperconvergence preview:

                          @gb.123 Hello,
                          The instruction in the first post are still the way to go ๐Ÿ™‚

                          I'm curious about that as well but the first post says that the installation script is only compatible with 8.2 and doesn't mention 8.3. Is that still the case or is the installation script now compatible with 8.3 as well? If not, is there an installation script that is compatible with 8.3?

                          I know that using XO is the recommended method for installation but I'm interested in an installation script as I would like to try to integrate XOSTOR installation into an XCP-ng installation script I already have which runs via PXE boot.

                          dthenotD 1 Reply Last reply Reply Quote 0
                          • dthenotD Offline
                            dthenot Vates ๐Ÿช XCP-ng Team @JeffBerntsen
                            last edited by

                            @JeffBerntsen That's why I meant, the way to install written in the first post still work in 8.3, the script still work as expected also, it basically only create the VG/LV needed on hosts before you create the SR.

                            J 1 Reply Last reply Reply Quote 0
                            • J Offline
                              JeffBerntsen Top contributor @dthenot
                              last edited by

                              @dthenot said in XOSTOR hyperconvergence preview:

                              @JeffBerntsen That's why I meant, the way to install written in the first post still work in 8.3, the script still work as expected also, it basically only create the VG/LV needed on hosts before you create the SR.

                              Got it. Thanks!

                              H 1 Reply Last reply Reply Quote 0
                              • H Offline
                                henri9813 @JeffBerntsen
                                last edited by henri9813

                                Hello,

                                I plan to install my XOSTOR cluster on a pool of 7 nodes with 3 replicas, but not all nodes at once because disks are in use.
                                consider:

                                • node1
                                • node2
                                • node ...
                                • node 5
                                • node 6
                                • node 7.

                                with 2 disks on each

                                • sda: 128GB for the OS
                                • sdb: 1TB for local sr ( for now ๐Ÿ˜„ )

                                I emptied node 6 & 7.

                                so, here is what i plan to do:

                                • On ALL NODES: setup linstor packages

                                Run the install script on node 6 & 7 to add their disks
                                so:

                                node6# install.sh --disks /dev/sdb
                                node7# install.sh --disks /dev/sdb
                                

                                Then, configure the SR and the linstor plugin manager as the following

                                xe sr-create \ 
                                    type=linstor name-label=pool-01 \
                                    host-uuid=XXXXย \
                                    device-config:group-name=linstor_group/thin_device device-config:redundancy=3 shared=true device-config:provisioning=thin
                                

                                Normally, i should have a linstor cluster running of 2 nodes ( 2 satellite and one controller randomly placed ) with only 2 disks and then, only 2/3 working replicas.

                                The cluster SHOULD be usable ( i'm right on this point ? )

                                The next step, would be to move VM from node 5 on it to evacuate node 5. and then add it to the cluster by the following

                                node5# install.sh --disks /dev/sdb
                                node5# xe host-call-plugin \
                                  host-uuid=node5-uuid \
                                  plugin=linstor-manager \
                                  fn=addHost args:groupName=linstor_group/thin_device
                                

                                That should deploy satelite on node 5 and add the disk.

                                I normally should have 3/3 working replicas and can start to deploy others nodes progressively.

                                I'm right on the process ?

                                aS mentionned in the discord, i will post my feedbacks and results from my setup once i finalized it. ( maybe thought a blog post somewhere ).

                                Thanks to provide xostor in opensource, it's clearly the missing piece for this virtualization stack in opensource ( vs proxmox )

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post