Hi everyone, I'm Eddie from Singapore. I run a small business that build mobile apps and websites for customers and we need a rock solid reliable Infrastructure to house and protect all our customers. I have been using XCP NG since the birth of it and have been early backers of XCP NG
.
@olivierlambert is my role model and I'm amazed with the growth of XCP NG under his leadership. In this post I want to share some simple setups that keeps my infrastructure rock solid since 2018.
This setup might not be suitable for your use case but it is pretty reliable and rock solid since day 1 and have survived 2-3 instances of disk failure. Before I start sharing my setup I would like to share what I tried previously and why I decided not to use them.
Previously this was my setup
Advantage : This setup is pretty good initially when there are no issues and you can move VMs easily around with ease. Easy maintenance of servers.
Disadvantages :
-
You rely greatly on your switch and san storage. If it fails, all your customers will go crazy
-
Storage maintenance is harder.
-
Your IOPS is limited to your SAN storage capability and Network speed.

Currently our setup uses the KISS approach. Keep it sweet simple approach
Each Nodes have local storage.
- We use NVMEs and SSDs without raid and local storage. Each node have 10g/25g
- We do 2 backups with 3 days retention. 1 Delta backups , 1 CR backup to spare servers. Truenas is a pretty good and reliable storag software. We also setup spare XCP NG servers with 8-16 local storage and replicate them.
- We use tools to monitor disk health.
- All our servers are 10-20% utilized.
What happens if a server / disk is failing
With the above setup, below are some easy steps to handle it .
- Step 1, Spin the replica from Replication.
- The backup from truenas is an additional backup.
This approach seems better for us as we believe that the infrastructure should be very easy for anyone to do the replacement and repairs. E.G CEPH Cluster and ZFS , NFS shared storage etc all requires an experienced engineer to repair them. With this setup and Xen Orchestra, we an easily restore broken vm/server easily and within mins it will be restored.