Warning: Using (K)PTI with (at least Debian Stable) PV Linux guests may cause trouble

  • Dear kind members,

    a quick word of warning after an unpleasant experience:

    • Debian Stable with linux-image-4.9.0-6-amd64 - 4.9.88-1+deb9u1 works flawlessly under XCP-ng 7.4.1

    • however, Debian Stable linux-image-4.9.0-7-amd64 - 4.9.110-1 enables (K)PTI (Kernel page-table isolation) which causes this Kernel to crash: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=903767

    • mitigation: either wait for 4.9.110-2 , to be released for Debian Stable soon or disable (K)PTI with: extra = 'elevator=noop pti=off'



  • XCP-ng Team

    Are you in PV or HVM mode for your Debian guest?

  • PV mode. It seems to be still faster than HVM.

  • XCP-ng Team

    Hmm doing what kind of load? On modern hardware, it's very likely that PVHVM is superior to purely PV.

  • Web, Mail, no HPC. 🙂 That's what I thought. It's a quite modern system: Dell R730XD with Intel Xeon E5 2600 v3 and Intel C610 series chipset.

  • Any suggestion on how to convert to PVHVM? I'll give it a try with clone/template VM. And then run some benchmarks. I am curious, too - whether it has changed. Last test I ran was with XS 7.0 as far as I remember. Most guests are PVHVM, but Debian I kept PV.

  • XCP-ng Team

    You just need to add the value "BIOS order" to HVM-boot-policy key. Eg:

    xe vm-param-set HVM-boot-policy="BIOS\ order" uuid=<YOUR_VM_UUID>

    That's it! Check the boot order then (cdn with c as disk, d as CDROM and n for network/PXE). Can be any combination like dcn.

  • XCP-ng Team

    Note: for a web server, expect +15% perfs on PVHVM vs PV. That's mainly due to how the RAM requests are handled: thanks to virt instructions, you remove the PV software layer to add/remove data in RAM (address translation is done by Xen in PV mode. In HVM, it's done by your CPU/chipset directly).

  • Great, thank you, Olivier. 🙂 I'll give it a try and will report back to you. Would be nice to finally also have Debian VMs running in PVHVM. CentOS 7.5 works flawlessly. It's about time then..

  • Converting a Debian VM from PV (paravirtualized) to PVHVM (PV-on-HVM drivers)

    Well, as often there's a little bit more to do than just running a oneliner. 😉 So I created a summary based on information found here, elsewhere and the one I added to successfully convert a Debian VM 9.x from PV to PVHVM on XCP-ng 7.4.x.

    • Within the Debian VM:

    Reconfigure GRUB defaults in /etc/default/grub (Debian). Make sure GRUB_TERMINAL is uncommented and set to console (disabling graphical modes, which caused a black screen in XenCenter / VNC for me):


    Update GRUB:


    Poweroff the vm:

    • Within the XCP-ng:

    Retrieve the UUID of the virtual machine:

    xe vm-list name-label=your_vm_name_goes_here params=uuid

    Set HVM boot mode:

    xe vm-param-set uuid=your_vm_uuid_goes_here HVM-boot-policy="BIOS\ order"

    Set local disk (c) and cdrom (d) as boot options:

    xe vm-param-set uuid=your_vm_uuid_goes_here HVM-boot-params="cd"

    Clear pygrub as boot loader:

    xe vm-param-set uuid=your_vm_uuid_goes_here PV-bootloader=""

    Clear the display arguments:

    xe vm-param-set uuid=your_vm_uuid_goes_here PV-args=""

    Find the UUID of the interface of the virtual disk:

    xe vm-disk-list uuid=your_vm_uuid_goes_here

    Set the disk device (VBD) as bootable:

    xe vbd-param-set uuid=your_vbd_uuid_goes_here bootable=true

    Start the VM and verify your VM actually is running in PVHVM mode:


    Hope this saves someone else a little bit of time. 🙂

  • @olivierlambert Now, that I have PVHVM it seems I want to have the latest PVH(v2) instead. 😄


    However, "PVH (v2) requires guests with Linux 4.11 or newer kernel." - and Debian 9.x runs on kernel 4.9.


    Too bad.. 😄

  • XCP-ng Team

    PVH mode is not support yet in XenServer/XCP-ng anyway. It's like a HVM but lighter (better name is "HVM-lite"). Basically (and in very short, so it's not a perfectly accurate description), it removes some bits of emulation to get rid of qemu in the middle, thanks to latest hardware improvements (in virt support).

    Anyway, PVHVM is still far better than PV for a lot of various loads.

  • @schories It should be noted here that the emulated BIOS in XCPNG7.6 seemingly doesn't care about the bootflags. If the first one (per device id) isn't bootable it will fail, even if xvdb is bootable! So if you e.g. have swap on xvda, move it to another device id at the end (and temporarily disable it in fstab or so)

Log in to reply

XCP-ng Pro Support

XCP-ng Pro Support