Pool best practices in a homelab setting?
-
Hi all, new to XCP-ng as a lot of others. Previously experienced with ESXi and pretty well versed with Linux (mostly Debian, but used a lot of different distros).
I'm moving houses and took that as an opportunity to jump ships from VMware after the whole Broadcom clusterf*ck, and reconfigure my homelab. I also got a new server – so going from one to two.
I've set them up in a pool, but I'm not sure if that is the best way – since they differ a bit i.e on network, and CPU/RAM generation.
Specs:
Dell Poweredge T420 (as NAS and some VMs):
- 2 x CPU Intel Xeon E5-2470 v2 @ 2.40GHz
- 96 GB RAM (12 x 8 GB, DDR3, 1333 MHz, ECC)
- 2 x EVO SSDs (for OS)
- 3 x Intel Enterprise SSDs (Fast storage, ZFS)
- 8 x Spinning rust for storage (Slow storage, ZFS)
- Dual 10Gbit nic
- Dual 1gbit nic
HP DL360 G10 (for VMs):
- 2 x CPU Intel Xeon Gold 6122 CPU @ 1.80GHz
- 64 GB (4 x 16 GB, DDR4, 2666 MHz, ECC)
- 2 x Intel Enterprise SSDs for OS / VM
- 8 x 1gbit (two 1gbit x 4 nics)
- Dual 10Gbit nic
Any advice?
-
@NoLooseEnds said in Pool best practices in a homelab setting?:
Hi all, new to XCP-ng as a lot of others. Previously experienced with ESXi and pretty well versed with Linux (mostly Debian, but used a lot of different distros).
I'm moving houses and took that as an opportunity to jump ships from VMware after the whole Broadcom clusterf*ck, and reconfigure my homelab. I also got a new server – so going from one to two.
I've set them up in a pool, but I'm not sure if that is the best way – since they differ a bit i.e on network, and CPU/RAM generation.
Specs:
Dell Poweredge T420 (as NAS and some VMs):
- 2 x CPU Intel Xeon E5-2470 v2 @ 2.40GHz
- 96 GB RAM (12 x 8 GB, DDR3, 1333 MHz, ECC)
- 2 x EVO SSDs (for OS)
- 3 x Intel Enterprise SSDs (Fast storage, ZFS)
- 8 x Spinning rust for storage (Slow storage, ZFS)
- Dual 10Gbit nic
- Dual 1gbit nic
HP DL360 G10 (for VMs):
- 2 x CPU Intel Xeon Gold 6122 CPU @ 1.80GHz
- 64 GB (4 x 16 GB, DDR4, 2666 MHz, ECC)
- 2 x Intel Enterprise SSDs for OS / VM
- 8 x 1gbit (two 1gbit x 4 nics)
- Dual 10Gbit nic
Any advice?
You may have some CPU capabilities suppression if they are going to be in a pool together. They're from different generations of CPUs as well as different years and on top of this the HP DL360 G10 is the Scalable form of the Intel Xeon processor.
So may help to if you can afford it to obtain a second identical HP DL360 G10 if you wish to operate two servers in a pool, as this will help when doing updates. As there are features which can be used when performing updates through Xen Orchestra, also useful when performing load balancing amongst other features.
Also a good idea to setup for a NAS functionality to choose between XO STOR when stabilised, if hyper converging. If going for shared storage and a NAS then True NAS, you can choose between True NAS Core or True NAS Scale. Though future innovations of True NAS will be based on the Scale based releases.
Also have a Management network and/or VLAN for the shared storage and the XCP-ng hosts, as well as a NAS network for the hosts which the shared storage can be accessed via. Then a VM network which the VMs can use to communicate with the rest of the Network including any clients.
-
Thanks. Not going to prioritise getting a new server, so I guess the recommended way on this hardware is to not use a pool? Keep them separated (used to standalone ESXi anyways, even if a pool seems cool).
Yes, running TrueNAS Scale now. Coming from Napp-IT on OmniOS. Ran that for years, and like it a lot, but the the UX is not up to par (understandably, since it's a one man team afaik behind Napp-IT). All pools imported fine. Having some weird permission issues on NFS, but that's a problem for another day.
-
@NoLooseEnds said in Pool best practices in a homelab setting?:
Thanks. Not going to prioritise getting a new server, so I guess the recommended way on this hardware is to not use a pool? Keep them separated (used to standalone ESXi anyways, even if a pool seems cool).
Yes, running TrueNAS Scale now. Coming from Napp-IT on OmniOS. Ran that for years, and like it a lot, but the the UX is not up to par (understandably, since it's a one man team afaik behind Napp-IT). All pools imported fine. Having some weird permission issues on NFS, but that's a problem for another day.
Each server is its own pool but they can't currently be used together without one dragging the other down in speed, and/or capabilities.
-
@john-c yes, I guess I should clarify that I meant "the same pool".
-
@NoLooseEnds said in Pool best practices in a homelab setting?:
@john-c yes, I guess I should clarify that I meant "the same pool".
@NoLooseEnds Should you reconsider there's https://www.bargainhardware.co.uk/refurbished-servers/hp-1u/hp-proliant-dl360-g10. They have a 10% discount for home lab users, contact them for the discount code.
I got my home lab servers from them, they are really professional and also helpful. Configured mine to the exact specification and also did a second identical one as well as can supply parts.
My home lab is based around a 3 Dell PowerEdge R620 servers, two are hypervisors and one is a shared storage server. The hypervisor hosts are both identically specified so there's no suppression of capabilities.
-
@NoLooseEnds A useful unique feature in Xen Orchestra is the capacity to utilise not just Maintenance mode on the host, but also on individual SRs for XCP-ng hosts.
Something which I came up with the idea for, during a brainstorming session in conversation with Vates during one of the Covid 19 years. I proposed it as a potential benefit for large deployments of XCP-ng and Xen Orchestra such as SME, Enterprises, Corporate and Government deployments.
Where there's likely many hosts and many storage servers at least some of which are going, are multiple ones connected to the same identical pools of servers. As well as different independent pools of servers.
The Xen Orchestra software is essentially something similar as an alternative to the VMware vCenter software, while each XCP-ng host is the equivalent to VMware ESXi.
Anyway both XCP-ng and Xen Orchestra are open source which people can contribute to in many different ways. Also there's XCP-ng 8.3 stable release coming, though its currently in beta.
With this new version its being worked on to include an embedded web interface XO Lite similar to how VMware ESXi from an update during VMware ESXi 6.0 and later gained a web interface for managing individual hosts.
Make sure to check out the Xen Orchestra (https://xen-orchestra.com) website or the appropriate page(s) on the https://vates.tech website.
-
@NoLooseEnds said in Pool best practices in a homelab setting?:
Thanks. Not going to prioritise getting a new server, so I guess the recommended way on this hardware is to not use a pool? Keep them separated (used to standalone ESXi anyways, even if a pool seems cool).
Yes, running TrueNAS Scale now. Coming from Napp-IT on OmniOS. Ran that for years, and like it a lot, but the the UX is not up to par (understandably, since it's a one man team afaik behind Napp-IT). All pools imported fine. Having some weird permission issues on NFS, but that's a problem for another day.
There's been some recent updates to TrueNAS Scale and also TrueNAS Core which is to enable NFS and Samba to interoperate as well as using new versions of OpenZFS. So depending on your versions of OpenZFS as well as NFS and/or Samba software you may be encountering some permission problems due to how TrueNAS structure's its data and permissions.
-
@NoLooseEnds The management network is over which the XAPI calls and SSH connections are made, these need to be isolated and kept from any internet connectivity (addressable), to protect from direct attack. In other words don't open any ports in the firewall of the network, pointing directly to the management network of the XCP-ng host(s) and the storage server. As well as give them internet visible host names as this will point black hats directly, at the hosts, this is really bad when with open firewall ports.
If you do not do this your practically begging for security issues (incidents) as this is a major part of how the XCP-ng hosts are managed, as well as carrying out actions on each other for the administrator (root). Remote internet based management of the XCP-ng host, and later multiple hosts are over the Xen Orchestra software. It's designed and secured for Internet based access, if you wish to make your hosts internet accessible for remote management by yourself from another location.
Oh by the way once everything is setup you can post your home lab (if you wish) on https://xcp-ng.org/forum/category/26/share-your-setup, several people have even posted photos of their home lab setup.
-
@john-c
Wow. What a "welcome" to a first post here. Thank you for all the information and you taking the time to write them. I'll write a better answer when I get a bit more time. Just wanted to say thank you.
