Testing ZFS with XCP-ng
-
@stormi this is for you
-
I knew we shouldn't have released on a Friday just before my holidays
XCP-ng community, note that I do that for you!
To answer the question, the zfs-tests package is not strictly needed so let's not install it (it pulls too many dependencies that we would have to add. We can consider making it installable later.). We forgot to add some steps in the procedure. We have updated it now.
If you run:
depmod -aThen you should be able to run
modprobe zfsThis should bring you at least a step further.
-
@eexodus If it's not working, we can chat to troubleshoot your issue
-
@borzel It would be nice also to improve https://github.com/xcp-ng/xcp/wiki/ZFS-on-XCP-ng-7.5-and-later because I wrote it and I never used zfs!
-
@stormi @borzel Thank you. I was able to create the zpool with those two extra commands. I then created a dataset and created a type=file and device-config:location SR. I realize ZFS is still a new feature, but is that the recommended way? I'm completely new to Xen so I am not familiar with creating SRs.
[root@xcp-ng-miqfcsgc] depmod -a [root@xcp-ng-miqfcsgc] modprobe zfs [root@xcp-ng-miqfcsgc] zpool create -f tank /dev/sda [root@xcp-ng-miqfcsgc] zfs create tank/sr [root@xcp-ng-miqfcsgc] xe sr-create host-uuid=MY_UUID name-label=zfs-sr type=file other-config:o_direct=false device-config:location=/tank/sr/zfs-sr -
@eexodus said in Testing ZFS with XCP-ng:
zpool create -f tank /dev/sda
If you have 4k disks (aka Advanced Format) use
ashift=12zpool create -o ashift=12 -f tank /dev/sdaI would say, for the first testing, please use:
zfs set sync=disabled tank zfs set compress=lz4 tank zfs set atime=off tankpool/discs can be live monitored with
zpool iostat -v 1 -
@borzel Not sure if this thread is the right place to ask .. but whats the min and recommended requirements to test ZFS ?
-
Well, depends on the level of perfs you want to achieve. A lot of RAM is the key.
-
@olivierlambert said in Testing ZFS with XCP-ng:
A lot of RAM is the key.
And if you want more performance, add more RAM
-
@stormi said in Testing ZFS with XCP-ng:
@borzel It would be nice also to improve https://github.com/xcp-ng/xcp/wiki/ZFS-on-XCP-ng-7.5-and-later because I wrote it and I never used zfs!
I have no write permissions
-
Good ressource with a lot of information, how to really use ZFS, build storages, tuning, etc.: https://www.zfsbuild.com
-
Zfs is a local sr and imho does not add too much advantages vs classic ext sr
The real value of zfs is to use built in replication to remote zfs system.
Proxmox has built a very interesting solutions around it: async replication of vm on a another node.I know xoa already offer a similar solution but is snapshot based and everyone knows the performance and size problem with this approach.
-
@buzzzo said in Testing ZFS with XCP-ng:
Zfs is a local sr and imho does not add too much advantages vs classic ext sr
meeeep
- ZFS uses (in standard configuration) half the RAM as read cache
- with
sync=disabledyour writes are faster (but synced to discs every 5 seconds; in one sequential batch - efficient use of slow spinning drives!) - it has checksums from fs-layer down to the device
- it's awesome
know xoa already offer a similar solution but is snapshot based and everyone knows the performance and size problem with this approach.
If we all had 48 hours a day and more people in the team, we could fully integrate ZFS into XCP-ng (with all bells and whistles). But for now we are happy to have the first step done - loose integrate ZFS to start using it and testing stability of the dom0 with ZFS in it.
-
@buzzzo Double meeeep
On file level SR (ext, NFS, local ZFS, XOSAN), there is no space issue doing snapshots. Also, perfs are good even with snapshots if you don't have an asthmatic SR.
-
@olivierlambert meeep!!!
-
Meep meep!
-
I tested latest and greatest XO (5.24.1 from sources) with latest XCP-ng 7.5 (incl. xcp-ng-extra repo/ZFS). Backups do not work
If the VDI is on a ext-SR, all is fine, backup working.
If the VDI is on a File-SR (ZFS based), the backup says "VDI_ERROR" or "VDI Chain to long". I will provide logfiles tomorrow if needed.Setup of my File-SR:
xe sr-create host-uuid=<xxx> name-label=zfssr type=file other-config:o_direct=false device-config:location=/zfspool/zfssr -
VDI chain too longdoesn't seems related. It means you have a VDI chain that's around 30 files.Can you provide the exact/detailed error log?
-
-
I searched in xensource.log, SMlog and syslog, no related entries found
How can I increase the loglevel of XO?
Here's the log output if I start this job again:
Aug 15 01:10:35 ubuntu xo-server[616]: 2018-08-14T23:10:35.241Z xo:xapi Snapshotting VM pmg as [XO Backup test] pmg Aug 15 01:10:36 ubuntu xo-server[616]: 2018-08-14T23:10:36.948Z xo:xapi exporting VDI pmg 0 Aug 15 01:10:41 ubuntu xo-server[616]: 2018-08-14T23:10:41.289Z xo:xapi Deleting VM [XO Backup test] pmg Aug 15 01:10:41 ubuntu xo-server[616]: 2018-08-14T23:10:41.585Z xo:xapi Deleting VDI pmg 0It's nearly nothing...
