XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Non-server CPU compatibility - Ryzen and Intel

    Scheduled Pinned Locked Moved Compute
    111 Posts 17 Posters 55.4k Views 16 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N Offline
      nhoty
      last edited by

      Hi everyone,

      I'm looking forward to building a homelab, with Ryzen 9 7900x. Do we have any information about non-server CPU compatibility from intel or Ryzen?

      A 1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        I think there's a non-negligible part of the community using Ryzen or Intel desktop CPUs.

        See https://www.youtube.com/watch?v=AbzKv_3x5S4

        N 1 Reply Last reply Reply Quote 1
        • N Offline
          nhoty @olivierlambert
          last edited by

          @olivierlambert Sure! I came to xcp-ng watching that video from @lawrencesystems , and asked him if he knows if am5 sockets from ryzen were working with xcp-ng he advises me to ask the forum.
          So I was hoping that someone has tried a new Ryzen CPU on their builds.

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by olivierlambert

            Ah, you mean on latest and greatest desktop AMD platform? I think you asked in the right place, we'll see if someone in the community did 🙂

            1 Reply Last reply Reply Quote 1
            • A Offline
              Andrew Top contributor @nhoty
              last edited by

              @nhoty I have an AMD Ryzen 5 5600U setup for testing in a small box. Not exactly the same but it works well. I would think the Ryzen 9 7900 in the right board would be good... For a server use ECC memory and just try to keep the CPU cool!

              As for Intel, I run some i5 and i7 desktop CPUs and they work faster than older server CPUs. Just note that Intel desktop CPUs normally skip the ECC options.

              N 1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                FYI, I ordered a small configuration to test it here. I'll keep you posted.

                1 Reply Last reply Reply Quote 0
                • D Offline
                  dj423
                  last edited by

                  Is it ok to mix hosts with Intel and Ryzen CPU in the same pool, or does that even matter? As pertaining to migrating VM instances between them? Thinking of refreshing my hosts to more modern CPU/RAM - but I don't want to induce issues of compatibility.
                  Thank you

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by olivierlambert

                    There's a fully automated system that will reduce all features to the lowest capable CPU of the pool, so there's no compatibility problems (a big thanks to @andyhhp for that)

                    1 Reply Last reply Reply Quote 0
                    • N Offline
                      nhoty @Andrew
                      last edited by

                      @Andrew Thank you I will try that, I'm only waiting for Asrock Rack Motherboard B650D4U (https://www.asrockrack.com/general/productdetail.asp?Model=B650D4U#Specifications ) so I can get ECC compatibility. Hope that they will release it soon.

                      1 Reply Last reply Reply Quote 0
                      • A Offline
                        andyhhp Xen Guru
                        last edited by

                        You cannot mix Intel and AMD CPUs in the same pool. They're simply not compatible enough for a VM to survive a migration between the two, so we explicitly disallow it. (You can't even --force a pool join to this effect.)

                        D 1 Reply Last reply Reply Quote 0
                        • D Offline
                          dj423 @andyhhp
                          last edited by

                          @andyhhp
                          Can we migrate VM's over to another pool with the different CPU type?
                          Thank you

                          A 1 Reply Last reply Reply Quote 0
                          • A Offline
                            Andrew Top contributor @dj423
                            last edited by

                            @dj423 Yes. You can do a cold (VM power off) migration between different CPU types.

                            1 Reply Last reply Reply Quote 0
                            • olivierlambertO Offline
                              olivierlambert Vates 🪐 Co-Founder CEO
                              last edited by

                              Warm migration is even a great solution in those cases (Intel to AMD or vice versa)

                              1 Reply Last reply Reply Quote 0
                              • olivierlambertO Offline
                                olivierlambert Vates 🪐 Co-Founder CEO
                                last edited by

                                Just installed XCP-ng 8.3 on a Zen 4 platform:

                                • Asus Prime B650M-Wifi
                                • Ryzen 5 7600 (Zen 4)
                                • 32GiB DDR5

                                Just had to enable x2API in the BIOS, everything worked out of the box (including the 2.5G Realtek NIC).

                                S 1 Reply Last reply Reply Quote 3
                                • M Offline
                                  mgales
                                  last edited by olivierlambert

                                  With the Ryzen 9 7900X, an Asus B650E series motherboard, and 32 GiB DDR5, the results I've gotten aren't favorable. I was able to install xcp-ng-8.3.testing-2023.02.15-12.19-install.iso without enabling x2API in the BIOS, and XCP-ng comes up fine. While xo-ce (running on the same box) comes up right away, it takes about 8-10 minutes before it acquires it's IP address (both XCP-ng and xo-ce use DHCP to get their IPs from a DHCP/DNS).

                                  After xo-ce is up and accessible, Windows 10 works well (install from ISO, boot & run performance is good). OSs from various Linux distros are horrendously slow. It took about 2 hrs to install Linux Mint 21.1 from an ISO. It takes several minutes, with any of the Linux ISOs I've tried (CentOS Stream 9.1, Ubuntu 22.04.1, Linux Mint 21.1) for a 2nd screen to appear after the initial install screen (which comes up fine).

                                  The install of Mint took about 2 hrs to complete.
                                  VM configuration: 8-CPUs, 8 GiB RAM, 22 GiB hard drive.

                                  Linux Mint OS comes up, but with similar slow boot time (29 minutes before LM logo, CPU usage at 100% on Stats tab during that 29 minutes, but oddly everything else at 0. Then I noticed CPU usage dropped and both Network and Disk throughput had activity, but still 0B of 8GiB RAM used, and I switched tabs and these 2 messages appear in the console screen:

                                  1. [  1.778229] vbd vbd-5696: 19 xenbus_dev_probe on device/vbd/5696
                                  2. [165.437239] piix4_smbus 0000:00:01.3: SMBus Host Controller not enabled!).
                                  

                                  (LM logo appeared on the console window shortly after the 2 messages above)

                                  At 48 minutes (per the VM_Started time on the Logs tab) the arrowhead cursor first appeared (console window), and the login screen appeared at 52 minutes. At 57 minutes the desktop is displayed (still at 0 GiB RAM usage) and system response is extremely slow.

                                  Another Note - these messages appeared on the xo-ce console window:

                                  [ 3684.425914] rcu: INFO: rcu_sched self-detected stall on CPU
                                  [ 3684.425914] rcu: #1-...!: (1 ticks this GP) idle=d22/0/0x1 softirq=3206/3206 fqs=1
                                  [ 3684.425914] rcu: rcu_sched thread starved for 4463 jiffies! g321 f0x0 RCU_GP_WAIT_FQS(5) -> state=0X402 ->cpu=1
                                  [ 3684.425914] rcu: #unless rcu_sched thread gets sufficient CPU time, OOM is now expected behavior.
                                  [ 3684.425914] rcu: RCU grace-period kthread stack dump:
                                  {Nothing after that on the screen}
                                  

                                  Side Note: my original Intel XCP-ng box shows all 4 (CPU, Memory, Network and Disk all peaking roughly at the same time that the Linux Mint OS boots, but on this new Ryzen box, 0B Memory at all times on the Stats screen for the Linux Mint VM. The xo-ce VM does show about 712 MiB Memory usage.

                                  I did try a separate boot, with X2API enabled in the motherboard BIOS before booting XCP-ng, but it didn't make any noticeable difference compared to having it set to Default.

                                  Repeating: I didn't experience sluggishness, problems installing, booting or running Windows 10 from an ISO under XCP-ng on this Ryzen box, only Linux distros.

                                  Far Side Notes: I used the same Linux Mint 21.1 ISO from above and installed it directly on a separate hard drive on the same Ryzen box and it works good. I'm running VMware running under Linux Mint 21.1 and Rocky Linux 9 loaded directly on this Ryzen box on separate hard drives and VMware works good when I boot on those drives (XCP-ng/xo-ce is running on a Samsung 970 plus M.2).

                                  I hope some of the information will be useful to the XCP-ng development team.
                                  I'd be glad to test a fix on my Ryzen box if that would help.
                                  It's a home lab box, so starting over isn't an issue.

                                  I'm fairly new to XCP-ng. I'm impressed by the support of the XCP-ng team for home users too.

                                  Addendum - inside the Linux Mint VM, I entered the free command in a terminal window.
                                  Results:

                                  	total	used	free	shared	buff/cache	available
                                  Mem:	8119596	727396	6087584	20004	1304616		7103828
                                  Swap:	1043340	     0	1043340
                                  
                                  1 Reply Last reply Reply Quote 0
                                  • olivierlambertO Offline
                                    olivierlambert Vates 🪐 Co-Founder CEO
                                    last edited by olivierlambert

                                    What kind of storage are you using?

                                    edit: I will try on my setup, but a least on Debian I had 0 issue and installation was blazing fast. Can you try on Debian and see if you still have issues?

                                    1 Reply Last reply Reply Quote 0
                                    • olivierlambertO Offline
                                      olivierlambert Vates 🪐 Co-Founder CEO
                                      last edited by olivierlambert

                                      So I installed Linux Mint 21.1 on my Zen4 setup, using a Debian 11 template, with 4vCPUs and 4GiB RAM and 40GiB virtual disk:

                                      • the ISO booted on the OS is less than 1 minute (likely 20 secs)
                                      • the installation itself was 5/6 minutes long tops

                                      After the install, booting the OS took around 15 seconds (to get to the login)

                                      Note: I'm using a cheap Kingston 120GiB SSD, not even an NVMe.

                                      edit: I had to do bash /mnt/Linux/install.sh -d debian -m 11 to install the tools, but that's it.

                                      1 Reply Last reply Reply Quote 0
                                      • olivierlambertO Offline
                                        olivierlambert Vates 🪐 Co-Founder CEO
                                        last edited by

                                        So it's likely an issue with your physical machine (buggy BIOS? thing not enabled?). I would start by the usual dmesg in the dom0, and also a xl dmesg.

                                        1 Reply Last reply Reply Quote 0
                                        • M Offline
                                          mgales
                                          last edited by

                                          Thanks @olivierlambert for going to those lengths to recreate the same steps that I had done.
                                          It's good to know that the new generation of Ryzen processors and motherboards should work without issues for most.
                                          I appreciate the time you spent - it's beyond what I expected.

                                          I'll try your suggestions with dmesg, experiment with BIOS settings, and do the install to a regular hard drive for starters. I'm not familiar with dmesg, but I'll read up on that - looks like it could reveal boot issues.

                                          It puzzles me why the only issues I'm having are with Linux VMs on my hardware, but hopefully I can figure out what's causing it. Forgot to mention that I had updated to the latest BIOS update from Asus before doing this install, but it didn't change the behavior from what I was experiencing prior to the update.

                                          If I'm able to get it working, I'll post back here what helped.

                                          • Mike ( @mgales)
                                          1 Reply Last reply Reply Quote 0
                                          • olivierlambertO Offline
                                            olivierlambert Vates 🪐 Co-Founder CEO
                                            last edited by

                                            Okay keep us posted 🙂 Since it's work here, it's not obvious on what to do, because not reproducing a problem makes it harder to solve 😞

                                            M 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post