XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Testing ZFS with XCP-ng

    Scheduled Pinned Locked Moved Development
    80 Posts 10 Posters 40.0k Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • olivierlambertO Offline
      olivierlambert Vates 🪐 Co-Founder CEO
      last edited by

      Your package is experimental, so feel free to add the modification inside it 🙂

      1 Reply Last reply Reply Quote 0
      • borzelB Offline
        borzel XCP-ng Center Team @nraynaud
        last edited by borzel

        @nraynaud said in Testing ZFS with XCP-ng:

        If other people can reproduce my results

        with the change in /etc/sparse_dd.conf I can copy my VMs from:

        • EXT3 -> ZFS
        • ZFS -> ZFS
        • ZFS -> EXT3

        Yeha! 🙂

        Thanks for your work!

        By the way, my XCP-ng replication host at work is working just fine with ZFS-SR. All stable like ZFS should be.

        1 Reply Last reply Reply Quote 1
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          Yay!! Thanks for testing 🙂

          1 Reply Last reply Reply Quote 0
          • E Offline
            eexodus
            last edited by eexodus

            I have a clean install of 7.5 and I'm following the guide on the wiki but can't install zfs-test or enable the zfs module:

            [root@xcp-ng-endlqfgb ~]# yum install --enablerepo="xcp-ng-extras" zfs-test
            Loaded plugins: fastestmirror
            Loading mirror speeds from cached hostfile
            Resolving Dependencies
            --> Running transaction check
            ---> Package zfs-test.x86_64 0:0.7.9-1.el7.centos will be installed
            --> Processing Dependency: lsscsi for package: zfs-test-0.7.9-1.el7.centos.x86_64
            --> Processing Dependency: ksh for package: zfs-test-0.7.9-1.el7.centos.x86_64
            --> Processing Dependency: fio for package: zfs-test-0.7.9-1.el7.centos.x86_64
            --> Processing Dependency: rng-tools for package: zfs-test-0.7.9-1.el7.centos.x86_64
            --> Finished Dependency Resolution
            Error: Package: zfs-test-0.7.9-1.el7.centos.x86_64 (xcp-ng-extras)
                       Requires: fio
            Error: Package: zfs-test-0.7.9-1.el7.centos.x86_64 (xcp-ng-extras)
                       Requires: lsscsi
            Error: Package: zfs-test-0.7.9-1.el7.centos.x86_64 (xcp-ng-extras)
                       Requires: rng-tools
            Error: Package: zfs-test-0.7.9-1.el7.centos.x86_64 (xcp-ng-extras)
                       Requires: ksh
             You could try using --skip-broken to work around the problem
             You could try running: rpm -Va --nofiles --nodigest
            [root@xcp-ng-endlqfgb ~]# zpool create tank /dev/sdb
            The ZFS modules are not loaded.
            Try running '/sbin/modprobe zfs' as root to load them.
            [root@xcp-ng-endlqfgb ~]# /sbin/modprobe zfs
            modprobe: FATAL: Module zfs not found.
            [root@xcp-ng-endlqfgb ~]#
            
            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              @stormi this is for you 🙂

              1 Reply Last reply Reply Quote 0
              • stormiS Offline
                stormi Vates 🪐 XCP-ng Team
                last edited by

                I knew we shouldn't have released on a Friday just before my holidays 🙂

                XCP-ng community, note that I do that for you!

                To answer the question, the zfs-tests package is not strictly needed so let's not install it (it pulls too many dependencies that we would have to add. We can consider making it installable later.). We forgot to add some steps in the procedure. We have updated it now.

                If you run:
                depmod -a

                Then you should be able to run
                modprobe zfs

                This should bring you at least a step further.

                1 Reply Last reply Reply Quote 2
                • borzelB Offline
                  borzel XCP-ng Center Team
                  last edited by

                  @eexodus If it's not working, we can chat to troubleshoot your issue 📰

                  stormiS 1 Reply Last reply Reply Quote 0
                  • stormiS Offline
                    stormi Vates 🪐 XCP-ng Team @borzel
                    last edited by

                    @borzel It would be nice also to improve https://github.com/xcp-ng/xcp/wiki/ZFS-on-XCP-ng-7.5-and-later because I wrote it and I never used zfs!

                    borzelB 1 Reply Last reply Reply Quote 0
                    • E Offline
                      eexodus
                      last edited by eexodus

                      @stormi @borzel Thank you. I was able to create the zpool with those two extra commands. I then created a dataset and created a type=file and device-config:location SR. I realize ZFS is still a new feature, but is that the recommended way? I'm completely new to Xen so I am not familiar with creating SRs.

                      [root@xcp-ng-miqfcsgc] depmod -a
                      [root@xcp-ng-miqfcsgc] modprobe zfs
                      [root@xcp-ng-miqfcsgc] zpool create -f tank /dev/sda
                      [root@xcp-ng-miqfcsgc] zfs create tank/sr
                      [root@xcp-ng-miqfcsgc] xe sr-create host-uuid=MY_UUID name-label=zfs-sr type=file other-config:o_direct=false device-config:location=/tank/sr/zfs-sr
                      
                      1 Reply Last reply Reply Quote 0
                      • borzelB Offline
                        borzel XCP-ng Center Team
                        last edited by

                        @eexodus said in Testing ZFS with XCP-ng:

                        zpool create -f tank /dev/sda

                        If you have 4k disks (aka Advanced Format) use ashift=12

                        zpool create -o ashift=12 -f tank /dev/sda
                        

                        I would say, for the first testing, please use:

                        zfs set sync=disabled tank
                        zfs set compress=lz4 tank
                        zfs set atime=off tank
                        

                        pool/discs can be live monitored with

                        zpool iostat -v 1
                        
                        A 1 Reply Last reply Reply Quote 3
                        • A Offline
                          AllooTikeeChaat @borzel
                          last edited by

                          @borzel Not sure if this thread is the right place to ask .. but whats the min and recommended requirements to test ZFS ?

                          1 Reply Last reply Reply Quote 0
                          • olivierlambertO Offline
                            olivierlambert Vates 🪐 Co-Founder CEO
                            last edited by

                            Well, depends on the level of perfs you want to achieve. A lot of RAM is the key.

                            borzelB 1 Reply Last reply Reply Quote 0
                            • borzelB Offline
                              borzel XCP-ng Center Team @olivierlambert
                              last edited by

                              @olivierlambert said in Testing ZFS with XCP-ng:

                              A lot of RAM is the key.

                              And if you want more performance, add more RAM 🙂

                              1 Reply Last reply Reply Quote 0
                              • borzelB Offline
                                borzel XCP-ng Center Team @stormi
                                last edited by

                                @stormi said in Testing ZFS with XCP-ng:

                                @borzel It would be nice also to improve https://github.com/xcp-ng/xcp/wiki/ZFS-on-XCP-ng-7.5-and-later because I wrote it and I never used zfs!

                                I have no write permissions 😐

                                stormiS 1 Reply Last reply Reply Quote 0
                                • borzelB Offline
                                  borzel XCP-ng Center Team
                                  last edited by borzel

                                  Good ressource with a lot of information, how to really use ZFS, build storages, tuning, etc.: https://www.zfsbuild.com 📰

                                  B 1 Reply Last reply Reply Quote 0
                                  • B Offline
                                    buzzzo @borzel
                                    last edited by

                                    Zfs is a local sr and imho does not add too much advantages vs classic ext sr

                                    The real value of zfs is to use built in replication to remote zfs system.
                                    Proxmox has built a very interesting solutions around it: async replication of vm on a another node.

                                    I know xoa already offer a similar solution but is snapshot based and everyone knows the performance and size problem with this approach.

                                    borzelB olivierlambertO 2 Replies Last reply Reply Quote 0
                                    • borzelB Offline
                                      borzel XCP-ng Center Team @buzzzo
                                      last edited by borzel

                                      @buzzzo said in Testing ZFS with XCP-ng:

                                      Zfs is a local sr and imho does not add too much advantages vs classic ext sr

                                      meeeep 🙂

                                      • ZFS uses (in standard configuration) half the RAM as read cache
                                      • with sync=disabled your writes are faster (but synced to discs every 5 seconds; in one sequential batch - efficient use of slow spinning drives!)
                                      • it has checksums from fs-layer down to the device
                                      • it's awesome 😉

                                      know xoa already offer a similar solution but is snapshot based and everyone knows the performance and size problem with this approach.

                                      If we all had 48 hours a day and more people in the team, we could fully integrate ZFS into XCP-ng (with all bells and whistles). But for now we are happy to have the first step done - loose integrate ZFS to start using it and testing stability of the dom0 with ZFS in it.

                                      1 Reply Last reply Reply Quote 0
                                      • olivierlambertO Offline
                                        olivierlambert Vates 🪐 Co-Founder CEO @buzzzo
                                        last edited by olivierlambert

                                        @buzzzo Double meeeep 😆

                                        On file level SR (ext, NFS, local ZFS, XOSAN), there is no space issue doing snapshots. Also, perfs are good even with snapshots if you don't have an asthmatic SR.

                                        B 1 Reply Last reply Reply Quote 0
                                        • B Offline
                                          buzzzo @olivierlambert
                                          last edited by

                                          @olivierlambert meeep!!! ☺

                                          1 Reply Last reply Reply Quote 0
                                          • olivierlambertO Offline
                                            olivierlambert Vates 🪐 Co-Founder CEO
                                            last edited by

                                            Meep meep!

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post