Please review - XCP-ng Reference Architecture
-
@nikade Thanks for your comments and thoughts. We're repurposing existing HP DL380 servers for the hosts, and was going to try repurpose our Nimble AF40 arrays, but they only do iSCSI, which means thick provisioning, which creates a capacity challenge for us (some of our VMs have been provisioned with 2-4TB virtual disks, but only using 100-300GB... so recreating smaller disks and data-cloning would be tedious but necessary).
TrueNAS is my 'gold prize', assuming it provides enough uptime and performance. Our IOPS and throughput requirements aren't huge; they only hit anywhere over 500MB/sec and a few thousand IOPS during backup jobs.
Replicating XOA is definitely a 'default'. But from my lab tests, redeploying and restoring config is to quick too, so I'm not too fussed about 'losing' XOA. I'd backup the config to on-premises 'remotes' and to cloud-based object storage.
Much appreciate your time and feedback, thank you!
-
We've used FreeNAS/TrueNAS for a long time and it works great, make sure you plan the sizing because that will really be important once you start putting some load on it (Backup time is really resource-consuming).
If you can, use SSD for the arrays instead of spinning rust - It will greatly improve performance.In our case we've used Dell R730xd's with dual Xeon hexacores (8 cores + HT each) and 128-256Gb RAM for ARC, then 2x200Gb SSD for SLOG and 1x200Gb SSD for L2ARC and the performance is OK.
We mostly do NFS shares (thin provisioning is important) but also some iSCSI LUN's mapped to VM's who need bigger disks, I really do not recommend mapping more than 400-500Gb VDI's directly from your SR.Our TrueNAS boxes are connected with 2x10Gb links but we really never see them do more than 1.4Gbit/s because of using RAIDZ2 and 10K disks (It is slow).
-
@olivierlambert Brief question please - would it make sense to install XOA on a dedicated computer (either 'from source' on Debian/Ubuntu, or as an only VM on a standalone XCP-ng host) so that it's managing pools but isn't adding load to the pool's compute / storage / network resources? Is there any recommendation here from Vates?
-
@TS79 I dont think it really matters, we run ours in one of our pools and we've been doing that since 2016 without any issues.
-
It's doable to dedicate an XCP-ng host for XOA. But XO doesn't use that much resources, so before it will be a "performance best-practice", I would argue it's a good thing for people with sensitive infrastructure where they want to split their mgmt environment to their prod environment. However, due to the level of isolation with Xen, it's doesn't matter in 90% of use case.
-
@olivierlambert Thank you - all makes sense
-
@nikade Thanks again for your input, much appreciated.
-
@TS79 said in Please review - XCP-ng Reference Architecture:
@nikade Thanks again for your input, much appreciated.
If your running TrueNAS Scale or TrueNAS Enterprise 24.04.2 as part of your deployment, with XCP-ng to replace VMware. Make sure you install TrueSecure app, as otherwise you'll be missing important security features on your TrueNAS.
-
@john-c said in Please review - XCP-ng Reference Architecture:
TrueSecure
Whats that? Never heard of TrueSecure on TrueNAS.
-
@john-c @nikade - I had to Google Search for TrueSecure, as hadn't heard of it before.
Seems good in that it's first-party solution, and security it typically always a good idea, but it's not really something for my use-case as a homelabber.
It mentions storage encryption: which to me immediately complicates things like deduplication, compression, and delta backups / replication.
TrueSecure seems to be positioned as a tool to achieve security compliance for strict standards like NIST / FIPS / government security regulations.
Still, good to know it exists and will be reading more about it for potential future advice! -
@TS79 said in Please review - XCP-ng Reference Architecture:
@john-c @nikade - I had to Google Search for TrueSecure, as hadn't heard of it before.
Seems good in that it's first-party solution, and security it typically always a good idea, but it's not really something for my use-case as a homelabber.
It mentions storage encryption: which to me immediately complicates things like deduplication, compression, and delta backups / replication.
TrueSecure seems to be positioned as a tool to achieve security compliance for strict standards like NIST / FIPS / government security regulations.
Still, good to know it exists and will be reading more about it for potential future advice!It's also where you can configure settings like minimum SMB protocol to use, SMB connection encryption and SMB connection signing.
-
@nikade said in Please review - XCP-ng Reference Architecture:
@john-c said in Please review - XCP-ng Reference Architecture:
TrueSecure
Whats that? Never heard of TrueSecure on TrueNAS.
It's an application or feature for TrueNAS Scale, TrueNAS Core and/or TrueNAS Enterprise. Which enables the enabling and configuration of security features of TrueNAS instances (software and/or hardware).
-
If you run backups outside of business hours, any impact on pool hosts cpu/memory performance is likely irrelevant (and limited by how many resources the XO is provisioned with anyway). The bigger potential impact is likely on your production storage, which again could be irrelevant outside of business hours.
However, if you want to perform backups more frequently and/or during business hours, in my experience the storage performance is the more likely to suffer noticeable impact. Unless your hosts are very highly utilized the additional cpu/memory load on a single VM shouldn't tip the scales. And at 500MB/sec your network shouldn't struggle either (I am assuming 10+Gbps links to get that speed).
And regardless of backing up during or outside business hours, or how long your backup window is, always consider the restore times! Performance of backup storage is always low priority until something needs to be restored
If your backup takes 8 hours your restore will take 8 hours. Or longer. Don't cut any corners on backup storage, it is very important! -
@john-c said in Please review - XCP-ng Reference Architecture:
@nikade said in Please review - XCP-ng Reference Architecture:
@john-c said in Please review - XCP-ng Reference Architecture:
TrueSecure
Whats that? Never heard of TrueSecure on TrueNAS.
It's an application or feature for TrueNAS Scale, TrueNAS Core and/or TrueNAS Enterprise. Which enables the enabling and configuration of security features of TrueNAS instances (software and/or hardware).
Alright - I didnt know that, thanks for the info.
