XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Guest UEFI Secure Boot on XCP-ng

    Scheduled Pinned Locked Moved Development
    25 Posts 12 Posters 14.4k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stormiS Offline
      stormi Vates 🪐 XCP-ng Team
      last edited by

      I'm opening this thread dedicated to the upcoming guest secure boot feature on XCP-ng, that some of you already started testing here.

      We will soon enter the final testing stages before release. I will post an answer to this thread when it's read, with all the details.

      apzA 1 Reply Last reply Reply Quote 1
      • stormiS Offline
        stormi Vates 🪐 XCP-ng Team
        last edited by

        The new updates that add Guest Secure Boot support are now available for testing.

        It brings SB support and also fixes the installation of update "KB4535680" on Windows server 2019.

        How to install on XCP-ng 8.2

        Install with:

        yum update uefistored varstored-tools --enablerepo=xcp-ng-testing
        

        No reboot required.

        Revert if needed with:

        yum downgrade uefistored varstored-tools
        

        Documentation

        The feature is documented here: https://xcp-ng.org/docs/guides.html#guest-uefi-secure-boot

        Feedback is welcome on the docs too: is it clear how to enable SB for VMs on XCP-ng?

        What you can test

        Check the docs for how to do concretely.

        • Pool setup for SB
        • Existing UEFI VMs still running well, no regressions
        • Enabling SB for a new VM
        • Enabling SB for an existing VM
        • Manipulating the certs for a given VM

        Linux and Secure Boot

        Due to certificate revocations last year and this year (caused by exploits in grub and/or shim), the current situation with Linux and SB is complicated. With the latest dbx (revocated certs) file from uefi.org, you won't be able to boot most distros with SB on.

        To make it work (which makes little sense until distros can sign new binaries again and publish updates, but is useful to us as far as testing is concerned), you can either install an older dbx, which leaves your VMs vulnerable to the latest grub/shim vulnerabilities. Or to use no dbx at all, which is even worse.

        I suggest to install the latest dbx pool-wide, and to modify the dbx variable on linux VMs that you want to enable SB on:

        • download an older dbx: https://uefi.org/revocationlistfile/archive
        • install it for a VM
          # Run this on an XCP-ng host.
          # "d719b2cb-3d3a-4596-a3bc-dad00e67656f dbx" is the identifier of the variable we'll set and 0x27 the attributes
          varstore-set {VM-UUID} d719b2cb-3d3a-4596-a3bc-dad00e67656f dbx 0x27 {name-of-dbx-file}
          
        J apzA 2 Replies Last reply Reply Quote 0
        • J Offline
          jhester7097 @stormi
          last edited by

          stormi

          I am not able to find the uefistored package. Any reason it wouldn't be showing up?

          [19:40 xcphost ~]$ yum search uefistored --enablerepo=xcp-ng-testing
          Loaded plugins: fastestmirror
          Loading mirror speeds from cached hostfile
          Excluding mirror: updates.xcp-ng.org
           * xcp-ng-base: mirrors.xcp-ng.org
          Excluding mirror: updates.xcp-ng.org
           * xcp-ng-testing: mirrors.xcp-ng.org
          Excluding mirror: updates.xcp-ng.org
           * xcp-ng-updates: mirrors.xcp-ng.org
          Warning: No matches found for: uefistored
          No matches found
          
          
          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Are you using XCP-ng 8.2, right?

            1 Reply Last reply Reply Quote 0
            • apzA Offline
              apz @stormi
              last edited by

              stormi Does this system change or break backups, either in XO or xe vm-export?

              olivierlambertO 1 Reply Last reply Reply Quote 0
              • apzA Offline
                apz @stormi
                last edited by

                stormi said in Guest UEFI Secure Boot on XCP-ng:

                Feedback is welcome on the docs too: is it clear how to enable SB for VMs on XCP-ng?

                Some test feedback:

                Documentation was clear enough to get it right on the first try.

                Tested on a WS 2019 virtual that has the KB4535680 issue, after enabling SB the updates installed without problems and a test clone of the virtual is now running isolated for a longer period test.

                1 Reply Last reply Reply Quote 2
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO @apz
                  last edited by

                  apz said in Guest UEFI Secure Boot on XCP-ng:

                  stormi Does this system change or break backups, either in XO or xe vm-export?

                  No it's not related at all.

                  1 Reply Last reply Reply Quote 0
                  • A Offline
                    ASUSEagle
                    last edited by ASUSEagle

                    stormi

                    I'm having issues with a Windows Server 2019 VM. I am running XCP-ng 8.2 with Xen Orchestra. After enabling secure boot the VM boots to Windows Recovery and will not boot to Windows Server. The VM will boot fine with secure boot off. These are the steps I followed is there something I am missing?

                    # yum update uefistored varstored-tools --enablerepo=xcp-ng-testing
                    
                    # secureboot-certs install default default default latest
                    
                    # varstore-sb-state d4960d10-e6dc-4bf7-daf4-5684c66cdb9e setup
                    

                    I then enabled secure boot through Xen Orchestra and started the VM.

                    Update:
                    There appears to be some problem with a driver signatures. The VM would not boot into safe mode but I was able to get it to boot by disabling driver signature enforcement. I've run sigverif.exe and it show all drivers are signed so I'm not sure what to do from here.

                    Also I forgot to mention earlier, the VM is set for UEFI firmware and is running the XCP guest tools. This is an existing VM that was setup with secure boot off I'm trying to turn it on so I can install KB4535680.

                    stormiS 1 Reply Last reply Reply Quote 0
                    • stormiS Offline
                      stormi Vates 🪐 XCP-ng Team @ASUSEagle
                      last edited by

                      ASUSEagle I realize that I forgot to mention it at the beginning of this thread: the guest drivers from XCP-ng are indeed signed in a way that does not please Windows when driver signature enforcement is on, which happens automatically when SB is enabled.

                      How did you disable driver signature enforcement? This would be a useful trick until we can provide new signed drivers (in progress).

                      This is an existing VM that was setup with secure boot off I'm trying to turn it on so I can install KB4535680.

                      I don't think you need to enable SB in order to be able to install KB4535680. In any case, it should not fail anymore.

                      A O 2 Replies Last reply Reply Quote 0
                      • A Offline
                        ASUSEagle @stormi
                        last edited by

                        stormi

                        I disabled it through advanced boot options. It does not persist through a restart so it has to be set every time the VM boots. The VM also took significantly longer to restart. It sits at the firmware splash screen for 30 minutes to an hour then boots the rest of the way in about a minute like normal. While sitting at the splash screen CPU0 utilization is hover around 90% while CPU1-3 are at 0%, memory is showing max at the 16GB I've assigned, and disk throughput is at 2-3 MiB(r) after an initial spike of 28 MiB(r).

                        These are the steps I took to boot with driver signature enforcement disabled:

                        • Once secure boot has been enabled, I start the VM. The VM fails to start Windows and boots to Windows Recovery.
                        • On the Windows Recovery screen I select "Troubleshoot".
                        • Then I select "Startup Settings", this brings up the "Advanced Boot Options" screen.
                        • I then key down and select "Disable Driver Signature Enforcement"

                        Once I've selected "Disable Driver Signature Enforcement" the VM restarts and hangs at the firmware splash screen as I described above before finally booting to Windows.

                        I didn't think I would have needed secure boot to install KB4535680 either but for some reason it would fail to install until I turned secure boot on. Thanks for the info on the guest drivers, I'll have to keep an eye out for when the signed drivers are released.

                        1 Reply Last reply Reply Quote 0
                        • J Offline
                          JurgenDM
                          last edited by

                          I confirm that installing this patch alone is not enough, also tried rebooting both the OS and XCP-NG.
                          Even after the the update the patch failes to install with error code 0x800f0922.
                          I've not tried enabling Secure Boot since it will probably cause issues with the installed guest drivers.

                          1 Reply Last reply Reply Quote 0
                          • O Offline
                            onyxfire @stormi
                            last edited by

                            stormi Do we have any timing or plans on getting the XCP guest drivers signed properly? This would be essential to ever being able to break fully away from citrix tools. Is there another thread tracking that progress and technical roadblocks?

                            1 Reply Last reply Reply Quote 0
                            • olivierlambertO Offline
                              olivierlambert Vates 🪐 Co-Founder CEO
                              last edited by

                              We are making progress on getting our EV certificates.

                              X 1 Reply Last reply Reply Quote 1
                              • X Offline
                                XCP-ng-JustGreat @olivierlambert
                                last edited by XCP-ng-JustGreat

                                olivierlambert stormi Here's a good link for persistent disablement of driver signature checking on Windows using bcdedit https://blog.pcrisk.com/windows/12194-how-to-disable-driver-signature-enforcement that may help those above wanting to use the XCP-ng drivers. If for some reason that doesn't work, they can, of course, use the signed Citrix drivers as a stopgap measure.

                                beshlemanB 1 Reply Last reply Reply Quote 1
                                • beshlemanB Offline
                                  beshleman @XCP-ng-JustGreat
                                  last edited by beshleman

                                  XCP-ng-JustGreat The issue I ran into was that bcdedit can't modify testsigning when secure boot is enabled, and setting testsigning before enabling secure boot resulted in an issue I can't quite recall, but I think it was a broken boot.

                                  N 1 Reply Last reply Reply Quote 0
                                  • N Offline
                                    noship @beshleman
                                    last edited by

                                    beshleman Have Vates fixed the problem with Secure Boot issue Yet

                                    stormiS 1 Reply Last reply Reply Quote 0
                                    • stormiS Offline
                                      stormi Vates 🪐 XCP-ng Team @noship
                                      last edited by

                                      noship Which issue are you talking about exactly?

                                      R 1 Reply Last reply Reply Quote 0
                                      • R Offline
                                        rjt @stormi
                                        last edited by

                                        stormi Installation of MS KB4535680 is failing for us as well as many others. To be clear, we have not downloaded the latest patches mentioned in XOA. Will simply installing the latest ca-certificates (dated Sept 14, 2021) and updated grub-efi (dated June 29, 2021) along with the other updates such as xcp-ng-release-config 8.2.0-8 fix this or do we have to manually make changes mentioned above as well?

                                        I expect having Windows reboot into UEFI and configuring the UEFI to attempt secure boot will be necessary no matter what. Meeting with a software vendor later today and they may ask why MS KB4535680 is not installed.

                                        stormiS 1 Reply Last reply Reply Quote 0
                                        • stormiS Offline
                                          stormi Vates 🪐 XCP-ng Team @rjt
                                          last edited by

                                          rjt You still need varstored-tools and uefistored from the testing repository.

                                          R 1 Reply Last reply Reply Quote 1
                                          • R Offline
                                            rjt @stormi
                                            last edited by

                                            Guest UEFI Secure Boot looks like a great guide. Everyone needs to read the "Boothole and fallouts" section.

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post