Warning: Using (K)PTI with (at least Debian Stable) PV Linux guests may cause trouble



  • Dear kind members,

    a quick word of warning after an unpleasant experience:

    • Debian Stable with linux-image-4.9.0-6-amd64 - 4.9.88-1+deb9u1 works flawlessly under XCP-ng 7.4.1

    • however, Debian Stable linux-image-4.9.0-7-amd64 - 4.9.110-1 enables (K)PTI (Kernel page-table isolation) which causes this Kernel to crash: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=903767

    • mitigation: either wait for 4.9.110-2 , to be released for Debian Stable soon or disable (K)PTI with: extra = 'elevator=noop pti=off'

    Cheers

    Alexander


  • Admin

    Are you in PV or HVM mode for your Debian guest?



  • PV mode. It seems to be still faster than HVM.


  • Admin

    Hmm doing what kind of load? On modern hardware, it's very likely that PVHVM is superior to purely PV.



  • Web, Mail, no HPC. 🙂 That's what I thought. It's a quite modern system: Dell R730XD with Intel Xeon E5 2600 v3 and Intel C610 series chipset.



  • Any suggestion on how to convert to PVHVM? I'll give it a try with clone/template VM. And then run some benchmarks. I am curious, too - whether it has changed. Last test I ran was with XS 7.0 as far as I remember. Most guests are PVHVM, but Debian I kept PV.


  • Admin

    You just need to add the value "BIOS order" to HVM-boot-policy key. Eg:

    xe vm-param-set HVM-boot-policy="BIOS\ order" uuid=<YOUR_VM_UUID>

    That's it! Check the boot order then (cdn with c as disk, d as CDROM and n for network/PXE). Can be any combination like dcn.


  • Admin

    Note: for a web server, expect +15% perfs on PVHVM vs PV. That's mainly due to how the RAM requests are handled: thanks to virt instructions, you remove the PV software layer to add/remove data in RAM (address translation is done by Xen in PV mode. In HVM, it's done by your CPU/chipset directly).



  • Great, thank you, Olivier. 🙂 I'll give it a try and will report back to you. Would be nice to finally also have Debian VMs running in PVHVM. CentOS 7.5 works flawlessly. It's about time then..



  • Converting a Debian VM from PV (paravirtualized) to PVHVM (PV-on-HVM drivers)

    Well, as often there's a little bit more to do than just running a oneliner. 😉 So I created a summary based on information found here, elsewhere and the one I added to successfully convert a Debian VM 9.x from PV to PVHVM on XCP-ng 7.4.x.

    • Within the Debian VM:

    Reconfigure GRUB defaults in /etc/default/grub (Debian). Make sure GRUB_TERMINAL is uncommented and set to console (disabling graphical modes, which caused a black screen in XenCenter / VNC for me):

    #GRUB_CMDLINE_LINUX="console=hvc0"
    GRUB_TERMINAL=console
    

    Update GRUB:

    update-grub
    

    Poweroff the vm:

    poweroff
    
    • Within the XCP-ng:

    Retrieve the UUID of the virtual machine:

    xe vm-list name-label=your_vm_name_goes_here params=uuid
    

    Set HVM boot mode:

    xe vm-param-set uuid=your_vm_uuid_goes_here HVM-boot-policy="BIOS\ order"
    

    Set local disk (c) and cdrom (d) as boot options:

    xe vm-param-set uuid=your_vm_uuid_goes_here HVM-boot-params="cd"
    

    Clear pygrub as boot loader:

    xe vm-param-set uuid=your_vm_uuid_goes_here PV-bootloader=""
    

    Clear the display arguments:

    xe vm-param-set uuid=your_vm_uuid_goes_here PV-args=""
    

    Find the UUID of the interface of the virtual disk:

    xe vm-disk-list uuid=your_vm_uuid_goes_here
    

    Set the disk device (VBD) as bootable:

    xe vbd-param-set uuid=your_vbd_uuid_goes_here bootable=true
    

    Start the VM and verify your VM actually is running in PVHVM mode:

    https://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers#Verifying_Xen_Project_Linux_PVHVM_drivers_are_using_optimizations

    Hope this saves someone else a little bit of time. 🙂



  • @olivierlambert Now, that I have PVHVM it seems I want to have the latest PVH(v2) instead. 😄

    https://wiki.xen.org/wiki/Xen_Project_Software_Overview#Summary

    However, "PVH (v2) requires guests with Linux 4.11 or newer kernel." - and Debian 9.x runs on kernel 4.9.

    https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg00540.html

    Too bad.. 😄


  • Admin

    PVH mode is not support yet in XenServer/XCP-ng anyway. It's like a HVM but lighter (better name is "HVM-lite"). Basically (and in very short, so it's not a perfectly accurate description), it removes some bits of emulation to get rid of qemu in the middle, thanks to latest hardware improvements (in virt support).

    Anyway, PVHVM is still far better than PV for a lot of various loads.



  • @schories It should be noted here that the emulated BIOS in XCPNG7.6 seemingly doesn't care about the bootflags. If the first one (per device id) isn't bootable it will fail, even if xvdb is bootable! So if you e.g. have swap on xvda, move it to another device id at the end (and temporarily disable it in fstab or so)


Log in to reply