XCP / XO and Truenas Scale or Core
-
I'm taking good advice from Olivier and we're setting up a NAS that should serve as backup storage, CR and storage for VM's - We're using 2 Dell Servers, 10GB lan and want to try Truenas as the storage manager.
I see there are 2 versions called Core and Scale, which from my understanding is Core is more a "traditional" storage platform whereas Scale offers additional tools more dedicated for virtualized environments and tools and is built on Linux instead of FreeBSD. It also however seem that Scale is not really ready for critical production and that leads me to believe there will be more frequent updates which could mean more problems down the line.
Has anyone here used either of the 2 with XCP / XO and are there any benefits in your opinion to which we should consider?
Our use case are as follows:
- NFS Storage for VM's that is replicated to a 2nd server
- Storage backups of VM's made in XO (not replicated)
Any tips would be appreciated. We don't intend on bring these into major production (some low level VM's so that we can test it properly)
-
@mauzilla Use core, it's more dedicated to just being a NAS.
Scale has kvm hypervisor and pod management features which would add unnecessary overhead when you just want NFS storage repository. -
Same, for storage purpose it's more logical to use a dedicated thing for… storage.
-
I've been running TrueNAS for hypervisor storage since version 9, it has been rock solid since. I cannot say quite the same about SCALE (though this was alpha).
If you just want storage and no extra services, go core, it'll do everything you need it to do. If you decide you need more, it's easier to go from Core -> Scale than the other way around.
-
Thank you all seems I've got my answer, will go core
-
TrueNAS Core is the end result from FreeNAS (what it was called originally), which was always based on FreeBSD. It’s been around for years and has a solid foundation as a NAS OS.
Core and Scale look a like and share some components but under the hood they’re very different. Scale hasn’t been out there for long. Due to being Linux based it has more main stream features and will probably get much more in near future and evolve quicker than Core. For pure NAS usage, definitely go with Core instead of Scale for now.
-
I have two truenas servers, main server only as a Nas running core and the second server as backup target for main servers' continous replication, cloud backup sync and a little testing and playing around running scale
they work well, and also work fine together.
both are stable and doing their job without issue.
For now the advantage of scale is the slightly better gui display (no more unused nics displayed in dashboard) and maybe more drivers available for more recent or uncommon components.
Core has the advantage of proven functionality and loads of knowledge all over internet -
Thank you, I opted to install Scale today but as murphy would have it, I did not know of the caveat of raid controller support only to find this just as I was about to setup the NFS mount. The servers I am using is Dell R720DX with H710 raid controller, and my understanding is that it's a no-go due ZFS and performance impact / possibility of data loss when using a raid controller. It's a bit of a logistical nightmare as I also don't feel comfortable in flashing vendor hardware with non-vendor firmware (just doesn't feel right for something meant to be in production) and I am 800km away from the cabinet so i've opted to take a chance until I find a suitable HBA card
HBA cards / config is a new venture for me, I take it I am not the first to walk down this line, any suggestions in HBA cards? It seems they push for a LSI 9300-8i and my understanding is it will not require any flashing but will use up a PCI-e slot, which is the last of my worries.
I also know I am completely venturing off the XO product line / discussion but maybe some other XO / XCP users run into the same scenario and seems that there are numerous XCP pro's in this thread already running truenas in production
-
@mauzilla You're correct on requiring an HBA, you want to avoid running ZFS on top of RAIDed disks. It really wants raw access to drives.
You mentioned not wanting to use a flashed card in production, totally get it. I did use this guide to get several working https://fohdeesha.com/docs/perc.html and use them in my R720xd system now.
If you opt for an LSI card, you'll eat up an HBA and you may need different cabling to accommodate the different length now.
-
@mauzilla said in XCP / XO and Truenas Scale or Core:
I opted to install Scale today but as murphy would have it, I did not know of the caveat of raid controller support only to find this just as I was about to setup the NFS mount. The servers I am using is Dell R720DX with H710 raid controller, and my understanding is that it's a no-go due ZFS and performance impact / possibility of data loss when using a raid controller. It's a bit of a logistical nightmare as I also don't feel comfortable in flashing vendor hardware with non-vendor firmware (just doesn't feel right for something meant to be in production) and I am 800km away from the cabinet so i've opted to take a chance until I find a suitable HBA card
FWIW I too have a R720 with a H710P RAID card. I was able to follow this guide https://fohdeesha.com/docs/perc.html to flash the card into "IT mode" and access the disks as HBA. It worked flawlessly and was fairly easy and quick to do. You can flash it back to RAID if needed.
My tests showed TrueNAS Scale worked well with the H710P in IT mode, seeing all the individual disks with no issue.
Eventually I decided to drop back to TrueNAS Core due to issues unrelated to the controller or disks.
-
@gsrfan01 I have opted to go for the firmware changes to IT mode and thankfully everything worked out perfectly. I have a question though, your R720DX, it's my understanding now that there are 2 additional 2.5 slots at the back of the server above the PSU. Are these slots "optional" or does the server come with the neccesary caddys / connections?
Reason I ask is we have purchased some SSD's to handle the workload, but have now installed them as part of our 12 drives in teh front of the server (and took out 2x 6TBs). I would obviously much rather move the SSD's to the back. Lastly, have you installed a SLOG device? I'm thinking of getting 2x PCIe NVME cards and installing 2 NVMe's for this but havent found a lot of info on the setup in the servers.
-
@mauzilla My R720XD came with the rear drives, they were optional from Dell during configuration so I try to get them included already when possible. You can grab the parts and install them yourself too.
They used to be a bit more expensive, looks like $50 now: https://www.ebay.com/itm/174894761780
They're "R720xd rear flex bays" if you wanted to look for them on not ebay, make sure if comes with the cables. I believe the cables run from the backplane to the rear flex bay on the R720.
-
@mauzilla Yea you definitely what to get it cable all together. It seem like if you forget the cable and decide to get it later usually its more pricy that way just due to separate shipping. I luckily got my server for $130 with it. Though it is missing CPU, RAM, CADDY. Every other components/specs was the highest possible for this R720XD configuration. I thought it was a steal as it was local and I was able to pick it up instead of shipping which is more than 50% of the price I pay for. If you live in a large city it seem like large corporation are renewing their server recently so they are dropping down in price to mostly $200-300 range typically. And they will keep on dropping over time.
Thanks for the reference to IT flashing. I saw that but wasn't sure if I should proceed but since a few here has done it I guess I shall try when time permit.
-
@mauzilla
Just an FYI, I have not tested the latest Scale 22.12 RC1 for performance but the previous versions of scale under some conditions was quite a bit slower. I have a video covering this topic here with the benchmarks and methodology used for testing.
