XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Micro XCP-ng cluster Thoughts and Suggestions.

    Scheduled Pinned Locked Moved Share your setup!
    16 Posts 3 Posters 1.9k Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • gskgerG Offline
      gskger Top contributor @brodiecyber
      last edited by

      @brodiecyber Personally, I think that the setup is very limited in terms of performance with RAM being the greatest challenge from the beginning.

      While I understand your initial idea of a low hardware and cost micro XCP-ng cluster to check out HA and learning runX/XOSTOR, you might be paying some good money for a low performance setup with little head room.

      Probably not an option, but if it is about learning and testing, you could go for a used mini pc (i10500t, 64GB RAM, 1TB boot and ISO drive, 2TB VM drive) and setup everything as a nested virtualization XCP-ng cluster? That would allow for snapshots and backups to make testing and rollback a breeze. My HP ProDesk 600 G6 is at an 20W average with 16 VMs running. Going for a physical XCP-ng cluster is much more fun though 😇 .

      brodiecyberB 1 Reply Last reply Reply Quote 0
      • brodiecyberB Offline
        brodiecyber @gskger
        last edited by

        @gskger I agree that the ram is the greatest challenge especially since dom0 takes 1.5 -2gb of ram in this setup.

        I wanted to do ServeTheHomes Tiny,Mini,Micro nodes but Ive tried looking for them secondhand and the cost of a single dell micro i5-8th gen were I live. I was able to get 3-j1900 motherboards off eBay including shipping.

        That's why when runX containers are more mature it would be a great platform for that. But apart form XCP-ng it works well as k3s nodes for kubernetes

        gskgerG 1 Reply Last reply Reply Quote 0
        • gskgerG Offline
          gskger Top contributor @brodiecyber
          last edited by

          @brodiecyber Ok, that makes sense.

          I once run XCP-ng on a HP T620 PLUS Thin Client with some lightweight Debian VM for a test. The HPs AMD GX-420CA has a bit higher CPU Mark compared to the j1900 (1787 vs. 1148), but the single thread performance is more or less the same.

          Nothing to run major workloads on, but should be ok for testing HA and general XCP-ng operations at low speed. No idea how XOSTOR will be performing on that platform, so please update us on your findings.

          brodiecyberB 3 Replies Last reply Reply Quote 0
          • brodiecyberB Offline
            brodiecyber @gskger
            last edited by

            @gskger

            Sure thing Ill update the post.

            Just waiting on a the cases and boot SSDs should be here Thursday. Then need to figure out if i want to use some cheap HDDs or cheap SSDs for XOSTOR

            1 Reply Last reply Reply Quote 0
            • brodiecyberB Offline
              brodiecyber @gskger
              last edited by

              @gskger But in the absence of runX containers

              I used some Alpine Linux VMs with an overkill amount of ram at 512mb yes mb and 10gb of storage and because it supports docker and compose you can run any docker app you just need to make sure their is enough ram for the running container.

              only thing is no VM icon

              1 Reply Last reply Reply Quote 0
              • brodiecyberB Offline
                brodiecyber @gskger
                last edited by

                @gskger Thought id update you the cases are coming today

                Apparently yesterday was a public holiday✌

                Great for relaxation.
                Bad for when your waiting on components.

                But on another not i had an idea
                I have an available PCIE 2.0 x1 slot that maxes out at the speed of SATA
                So i had the idea for using an m.2 to PCIE adapter and some cheap nvme drives
                so i can have a local SR on each node and an HA SR distributed on all nodes. not sure yet but this could be interesting

                1 Reply Last reply Reply Quote 0
                • brodiecyberB Offline
                  brodiecyber
                  last edited by

                  1682689286731.jpg New Hosts.PNG

                  Hosts are online and updating before i cluster them

                  brodiecyberB 1 Reply Last reply Reply Quote 1
                  • brodiecyberB Offline
                    brodiecyber @brodiecyber
                    last edited by

                    @brodiecyber After 2 hours of waiting its fianlly updates and cluster configured.

                    I will note out of 8GB of RAM. Dom0 is using 1.9GB leaving me with 6.1GB of RAM

                    1 Reply Last reply Reply Quote 0
                    • brodiecyberB Offline
                      brodiecyber
                      last edited by

                      Update the cluster is doing good i just need to get some SSDs for XOSTOR.

                      I have a 1cpu 1gb ram 25gb storage VM running over gigabit networking for ha storage. i don't recommend it its just for testing because disk io and bandwidth is limited.

                      VM runs and live migration takes about 17 Mississippi😁

                      brodiecyberB 1 Reply Last reply Reply Quote 0
                      • brodiecyberB Offline
                        brodiecyber @brodiecyber
                        last edited by

                        @brodiecyber

                        Update Setting up cloud-init templates to provision VMs with Terraform and Ansible for application and patch management.

                        Maybe ill even setup a XO-ce dedicated to this cluster 🤠

                        MicroXen.PNG

                        brodiecyberB 1 Reply Last reply Reply Quote 0
                        • brodiecyberB Offline
                          brodiecyber @brodiecyber
                          last edited by

                          @brodiecyber ok

                          new update

                          Redoing my cloud-init templates and preparing for the next version of Debian in June release 12.
                          Am using terraform to provision virtual machines and cloud-init and ansible to install software

                          I'm also using the boot disk as VM storage i don't recommend it as its a 120GB disk and i had to redo my templates as the VM they provision are to big for the little storage.

                          but Im running

                          • HA pihole with a keepalived

                          • WIreguard

                          • Terraform

                          • Dashboards. RSS, Archivebox

                          Will be adding

                          • dedicated XO-ce VM

                          • Unifi Controller

                          Infrastructure.PNG

                          1 Reply Last reply Reply Quote 0
                          • brodiecyberB Offline
                            brodiecyber
                            last edited by

                            Ive still got 4GB of ram to spare on Xen- 2 and 3 so abit more room for 1 CPU/RAM vms to deploy

                            I still think that this a the perfect environment for RUNx containers just the efficiency especially with memory will help alot

                            1 Reply Last reply Reply Quote 0
                            • brodiecyberB Offline
                              brodiecyber
                              last edited by

                              New update

                              I got a asus n3150i-c motherboard and found a spare ssd
                              P_setting_fff_1_90_end_500.jpg

                              Im reusing a node for parts and am installing truenas scale as a shared storage target for NFS

                              Its only 100gb of SSD storage but before i redo my workloads im waiting on debian 12 to be released

                              so im down a compute node but am up on shared storage with dedicated node

                              1 Reply Last reply Reply Quote 0
                              • olivierlambertO olivierlambert moved this topic from Community on
                              • First post
                                Last post