When you miss 10G in your homelab
-
Thought this is the best place to contemplate on how I miss 10G in my small form factor homelab (two HP Elitedesk 600 G6 Mini).
Moving all VM disks to local storage for maintenance work on the shared storage (Synology RS818+) takes a very long time, even at 117 MiB/s (123 MB/s). And I can now spell TOO_MANY_STORAGE_MIGRATES(3) backwards
.Just kidding
. First time in about two years I need to do this. -
I'm still running 1G in my homelab. Although I do have plans on finally hitting 10G this year.
-
@gskger I have 2.5G on my HP Elitedesk 800 G3 but I need to manually install a 2.5G into the wifi slot
about $10 for one nowaday. Especially now that XCP-ng support 2.5G i am so much more excited. -
@gskger Found it this was the one i use for HP800 g3 mini. Only if you are ok with realtek card the intel cost quite a bit more but it works as i used it and left it on for months already.
link
-
Yeah don't think I could live without 10G in my lab lol.
-
@wilsonqanda Thank you for sharing your positive experience with the Realtek 2.5G NIC. I am not sure if 2.5G is just a step in between on the way to 10G, which is why I hesitate to invest in 2.5G technology. But I would probably get away with less than €100 for two NICs and a cheap switch plus another NIC for the shared storage.
ServeTheHome has an interesting post on the HP Flex IO V2 10Gbase T Module, but they are far too expensive at 235€ atm.
-
@gskger Lol yea price wise it is just 10X more but another issue is that 10G is not going to help in the hp800 g6 or in my particular case g3 is too limited. But considering I am able to build everything and run the server under $45 with penny in electricity.
-
@gskger Yea the step in between is hard to justify. Esp. in my case since I have both 10G for my R720XD as well but these large servers are just too loud to be in the same room. lol
. I end up not wanting to not hear these severs and reducing electrical cost.FYI I used the one I show and used an old credit card and drill two holes in the card mount the 2.5G and insert it right in the display port I used to have on the HP800g3 lol. Fit perfect and work like a charm.
-
@gskger XAPI may not get much above 2Gbps, so it is possible you would not get better performance with 10Gbps. Of course, that is looking at just raw bits-per-second, not factoring in all the other performance features on a real server class nic.
-
2.5G is too slow for shared storage IMO. You end up sharing half SATA SSD speeds across all VMs one the host. I'm thinking about trying some 10G USB-C adapters but even then I'm not sure I want to use shared storage.
-
@rjt Thats true. I once tested VDI host-to-host disk migration on an older low tech 10G setup (two Dell Optiplex 9010 with SSD and an Intel X520-DA1 NIC, Mikrotik CRS305-1G-4S+IN) and did not get much higher bandwidth. This may be faster with three simultaneous disk migrations.
-
@gskger Just wondering since we are on topic of storage and network. What would you guys do to make a website highly available?
My understanding is:
- Nginx or Apache to run frontend (doesn't change much) so XCP-ng can host these VMs pretty much static beside the changes to the database and the contents that is in the storage. So as long as there is a reverse proxy and load balancer it should be good.
- Database - (Mariadb with HA active-active enable) Change all the time and needs to be geo-replicated and believe MariaDB does this by itself.
- Storage - Ceph active-active sync. Maybe async work not sure... (Ceph storage best for this as it has active-active sync.)
Does that sound about right or i am missing too much detail?
-
@wilsonqanda Using HA adds more complexity (to XCP-ng and front-/backend systems). Your use case should really require HA to justify that complexity, especially if you add geo-replication to the mix. Most scenarios can probably be addressed with incremental replication for a low recovery point objectiv (RPO) and a good recovery time objective (RTO), but your mileage my vary.
