Nested Virtualization of Windows Hyper-V on XCP-ng
-
@Chuckz Why do you need Core Isolation enabled in a VM? Core Isolation is designed to protect processes within Windows 11 by using VBS, if you're already isolating the VM I don't see a huge reason to have it enabled.
It's worth noting again that Microsoft themselves says to NOT use Nested Virt for production use, very specifically in their own documentation.
I get what you're wanting here but reality is 99% of places don't need nested virtualization and if they do they should probably rethink it since it's not considered stable or production ready on ANY hypervisor. This isn't specific to XCP-ng.
Hyper-V has probably the best nested virt support and even they say it should not be used in production environments.
I'm not saying I don't want this feature to work better, I do. But I can't imagine it should be a priority for Vates or anyone working on Xen because it's not really needed for production setups.
If I am missing some reason you have to have this enabled please let me know, but virtualizing Windows just to nest another Windows so you can enable Core Isolation is really cumbersome and not worth any benefits it provides as far as I can tell.
-
@Chuckz Why do you need Core Isolation enabled in a VM? Core Isolation is designed to protect processes within Windows 11 by using VBS, if you're already isolating the VM I don't see a huge reason to have it enabled.
It's worth noting again that Microsoft themselves says to NOT use Nested Virt for production use, very specifically in their own documentation.
I get what you're wanting here but reality is 99% of places don't need nested virtualization and if they do they should probably rethink it since it's not considered stable or production ready on ANY hypervisor. This isn't specific to XCP-ng.
Hyper-V has probably the best nested virt support and even they say it should not be used in production environments.
I'm not saying I don't want this feature to work better, I do. But I can't imagine it should be a priority for Vates or anyone working on Xen because it's not really needed for production setups.
If I am missing some reason you have to have this enabled please let me know, but virtualizing Windows just to nest another Windows so you can enable Core Isolation is really cumbersome and not worth any benefits it provides as far as I can tell.
Hi @planedrop Sure, I agree the isolation that XCP-ng's Xen hypervisor provides protection of other guests on the host from a compromised Windows guest, but it won't protect the confidential data and other services provided by that compromised Windows guest. Nor will it protect the other hosts on the network from attacks that can spread laterally via the network rather than via security holes between different guests on the host. The hypervisor is only one piece of the security process, it is not enough. So I want to protect that Windows guest from attacks that could have been stopped with core isolation, because I don't want even a single device or guest in my infrastructure compromised.
-
I'm not saying I don't want this feature to work better, I do. But I can't imagine it should be a priority for Vates or anyone working on Xen because it's not really needed for production setups.
I respectfully disagree. This should be a priority for Vates and Xen because currently, neither Vates nor Xen can provide the core isolation feature for their Windows guests. I think it is a mistake to think the isolation that Xen / Vates / XCP-ng provides between guests on a XCP-ng host can provide the same level of security for a Windows guest that the core isolation feature that nested Hyper-V can provide for that Windows guest.
-
@Chuckz wouldn't the better solution here be to prioritize making Core Isolation work within XCP-ng guests rather than focusing on nested virtualization?
Nested virt has other issues and again should not really be used with high priority VMs.
I guess that's the whole thing I'm getting at, nested virt isn't the fix for this specific issue.
-
That's interesting. Microsoft's Enterprise Connected Cache is a containerized setup on Linux containers, however the installer is only available for Windows. Running Linux-based containers requires nested virtualization.
This is what got me wondering about nested virtualization support for XCP-ng.
-
@Byte0 I think this is a fair use case, containers are a bit different and what you described is basically how containers work on most setups. I mean K8s on XCP-ng works that way, you deploy VMs which then have containers running inside them.
So not quite the same as nested virt.
-
@Chuckz wouldn't the better solution here be to prioritize making Core Isolation work within XCP-ng guests rather than focusing on nested virtualization?
Nested virt has other issues and again should not really be used with high priority VMs.
I guess that's the whole thing I'm getting at, nested virt isn't the fix for this specific issue.
@planedrop That would be cool if core isolation could work directly in XCP-ng guests without help of nested Hyper-V. I already asked @stormi if this is possible in earlier post today:
Chuckz said:
Does XCP-ng support core isolation in Windows 11 guests via some mechanism other than by nested virtualization? I presume that I cannot get core isolation to work in Windows guests because of lack of NV support in Xen.
So far no reply to that question but I only asked earlier today. I presume it is a complex technical question that cannot be answered without discussion with upstream Xen developers.
-
What's different in this situation is that if I install Microsoft Enterprise Connected Cache on a VM, that Windows VM requires nested virtualization to run a Linux container. That means nested virtualization is required.
Linux containers on a Linux VM do not require nested virtualization.
-
Hi @Byte0 I think that type of cached content from Microsoft would only be needed in organizations that run lots of Microsoft tech where there would be at least some Windows Servers or Windows 11 devices running on bare metal where the cached content could be deployed, rather than trying to deploying them in a Windows guest running on XCP-ng where lack of nested virtualization could be a problem.
-
The Microsoft Enterprise Connected Cache node is intended to be installed on a Windows Server where the Windows clients use that cache server.
Since XCP-ng cannot do nested virtualization, you are correct: the use Microsoft Enterprise Connected Cache is not available unless a separate server running Windows bare-metal or Hyper-V (or ESXi, ...) with nested virtualization enabled.
Most places use a hypervisor on all bare-metal servers and virtualize the compute. That was my situation. Which means I was unable to improve InTune Win32 app deployment, because we had switched to XCP-ng.
-
The Microsoft Enterprise Connected Cache node is intended to be installed on a Windows Server where the Windows clients use that cache server.
Since XCP-ng cannot do nested virtualization, you are correct: the use Microsoft Enterprise Connected Cache is not available unless a separate server running Windows bare-metal or Hyper-V (or ESXi, ...) with nested virtualization enabled.
Most places use a hypervisor on all bare-metal servers and virtualize the compute. That was my situation. Which means I was unable to improve InTune Win32 app deployment, because we had switched to XCP-ng.
I think possibly a solution for XCP-ng would be something like runx, which is a technology being developed by XCP-ng in Tech Preview and allows for running containers based on OCI format. This is advertised as supporting Linux containers, but I don't know if it also supports Windows containers. XCP-ng could benefit with a technology like that which clearly also supports Windows containers where possibly Connected Cache node containers could be deployed.
https://xcp-ng.org/blog/2021/09/14/runx-next-generation-secured-containers/
-
That's great and very cool.
Unfortunately Microsoft doesn't publish the containers in a place like docker hub, rather they distribute a PowerShell script that installs and configures the containers on a Windows server, specifically. Updates to the Microsoft Enterprise Connected Cache is also updated via Windows services/tools.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login