Minimums for XOstor disk configuration?

Greg_E

Just wondering about a possible XOstor configuration as far as performance:

3 hosts, each can have 3 spinning disks for HCI. Need about 6tb of usable free space and understanding each host is essentially a mirror of the other hosts. Total 12 drives in the pool.

What size drives would be best to achieve this? Would 2tb x3 drives give me this because they are not put into a raid like array on each host? Would I need to go 4tb x3 drives because it would make a RAID3 like array on each host? Or would I need larger x3 drives in each host? I only have SATA on these hosts.

Also how close are we to a "from sources" guide this would certainly get prototyped in my lab with a different drive configuration (single nvme), but I'd need to plan ahead for production. Time always being against me, it would take months to get this prototyped and the demo period will run out multiple times while I'm trying to work on this.

An example of time against me, I've had Harvester up for a month and still haven't build the first VM and haven't tried to get Rancher installed on a docker and integrated. It's certainly a much heavier lift to get to where XCP-ng and XO get you. I just feel like eggs in one basket is not something I should be doing (looking at you Broadcom and IBM ).

Pilow

@Greg_E hi there,

beyond number of minimal hosts to be supported (think it's 3) and minimal disks to get good redundancy (I think its minimal 3 per host, must be identical) you have a replication parameter when building an XOSTOR

it defaults to two (you have two copies of each workload) and this parameter can impact your total usable space.

also beware of network requirements (for satellites connections, and DRBD replication)
minimum of 2 nics per server, and DRDB replication should be at least 10Gb nics

tip : the linstor-controller is not always the pool master...

go here : https://docs.xcp-ng.org/xostor/#prerequisites

DustinB

@Greg_E For only 6TB usable, why would you use spinning rust? Get SSD, the price delta is going to be minimal and the performance and reliability will be far higher.

Greg_E

@DustinB

Indeed I will, but are the prices really that close? I was looking a few weeks ago with some replacements and spinning was still enough cheaper that it made a difference. I was looking for around 4tb enterprise class drives.

DustinB

@Greg_E The MTBF for Enterprise Hard drive, Enterprise SSD and Consume SSDs really falls into "it doesn't matter" category IMO.

https://www.backblaze.com/cloud-storage/resources/hard-drive-test-data
https://www.serverstor.com/samsung-pm9a3-vs-990-pro-enterprise-vs-consumer-ssd-showdown/
https://www.ontrack.com/en-au/blog/kingston-blog-enterprise-and-client-ssds-compared-part-2

Since most hypervisors aren't getting slammed constantly with read/writes ops even something like a "decent" consumer set of SSDs will last effectively as long as the Enterprise Hard Drives if not slightly longer.

The break down above really helps to explain it all.

Enterprise HDDs -1.2-1.5 million hours between failure
Enterprise SSDs - 2-3 million hours between failure
Consumer SSDs - 1.2 - 2 million hours between failure

Unless you expressly need some specific feature of Enterprise hardware (ECC Ram etcera) why spend more on it?

It's slower, as costly if not just more/less than the SSD comparable, adds risks during recovery time after a failure (because it's slower to rebuild an array on spinning rust).

Really the only things I would consider today would be Enterprise SSDs or Consumer SSDs.

DustinB

And to really round this out, the MTBF for any of these is in the millions of hours (1.2-3M), that's a use time of 136.968 - 342.46 years respectively.

Basically, if a drive dies, just replace it no matter what, but in the end the reliability of these drives is meant to outlast all of us.

Unless you actually need some specific function provided in some form-factor or model, don't bother.