Nested Virtualization of Windows Hyper-V on XCP-ng
-
@planedrop Here's hoping! Though I'm not hopeful because their installer application hooks into Hyper-V and creates an Ubuntu server on there and then extract tarballs into it... It's very strange.
-
@FTSSupport That's an interesting way to do it instead of just having an OVA file lol.
I'm honestly a little surprised any vendors require Hyper-V too, like, if you're going to require something, why not use the industry standard that is ESXi?
And if that was the case, it would be an OVA which would be something you could natively import to most hypervisors anyway.
What an interesting situation lol.
-
@planedrop Thought you might be interested to hear my results... The Vendor A) refuses to talk to me since I'm an MSP and not a direct employee of my client... Stupid, but I sent questions to my clients to give them...
They have straight-up refused to discuss trying anything outside of Hyper-V and completely acted stupid when I mentioned something about moving the VHD that their software creates in Hyper-V and put it in our hypervisor directly... Said that's unsupported and we will not assist you.
Still trying to figure out how to get around this... I had Hyper-V running on a Proxmox VM but the software seems to stall out and never finish installing. I'm trying XCP-NG 8.2 but based on all the conversations here, I'm assuming Hyper-V will refuse to run.
-
@FTSSupport Yeah that's really annoying, oof, sorry to hear that.
Maybe the only option is going to be getting a new physical host and then using Hyper-V on that? Hate to say it, but sounds like it could be the final result.
I did that with ESXi for this one vendor since I didn't have a choice at the time, it was lame, but the company understood the need for the expense, though it helped that we were going to need another (albeit not as powerful) host anyway.
-
Serious movement appears to be happening with respect to NV. See videos below cross-posted from this forum thread:
Nested Virtualization (X86) Part I - George Dunlap, Xen Server:
https://www.youtube.com/watch?v=8jKGYY1Bi_oNested Virtualization (X86) Part II - George Dunlap, Xen Server:
https://www.youtube.com/watch?v=3MxWvVTmY1s -
@stormi said in Nested Virtualization of Windows Hyper-V on XCP-ng:
Actually, Xen never officially supported Nested Virtualization. It was experimental, and broke when other needed changes were made to Xen. Now there's work to be done to make it fully supported, and this won't happen before the final release of XCP-ng 8.3. This will be documented in the release notes.
This is also an issue for us internally as we create a lot of virtual pools for our tests.
I read through a lot of the earlier posts and finally started scrolling to find this, which is the answer I was looking for. Why do I care? There is a Microsoft evaluation learning lab for things like Intune that runs in Hyper-V, basically a bunch of VHD (x) that get spawned as needed. Applications I need to teach myself. Running XCP-NG 8.3 current updates for this lab.
If it doesn't happen, then I'll just need to throw an eval version of Windows Server on something else like an HP T740 to run these labs, not the biggest issue for me.
Link for the labs if anyone is curious (free with an email registration like all the evals):
https://www.microsoft.com/en-us/evalcenter/evaluate-mem-evaluation-lab-kit
I'd think direct Docker support would be a higher priority than nested virtualization with a focus on Hyper-V. But that's just me.
-
Serious movement appears to be happening with respect to NV. See videos below cross-posted from this forum thread:
...
@XCP-ng-JustGreat Well after over a year and a half and AFAICT not much progress from upstream on NV support for Windows/Hyper-V in Xen and XCP-ng. It is discouraging that support from upstream for this feature has not come yet. So I think we, as users of XCP-ng and Xen who are interested in this feature could roll up our sleeves and start working on the problem again and hopefully jump start the process of getting this feature working in upstream Xen and XCP-ng. I will follow up with another post to propose what users of Xen and XCP-ng can do to help. -
@Chuckz Yeah it would be a nice feature to see. I think the issue though is how much work it takes when it's not something anyone should be using in production. It's really just a heavy homelab feature.
I want it to work, don't get me wrong, but no big org should be doing nested virt, it's just not a good idea and even Hyper-V recommends against it.
-
@Chuckz Yeah it would be a nice feature to see. I think the issue though is how much work it takes when it's not something anyone should be using in production. It's really just a heavy homelab feature.
I want it to work, don't get me wrong, but no big org should be doing nested virt, it's just not a good idea and even Hyper-V recommends against it.
For now, I think you are right except for Windows-centric shops. Going forward, there is no doubt that running Windows will be only on Hyper-V unless third party hypervisors can maintain support for the increasing number of features in Windows that rely on NV support. For example, important security features in Windows 11 such as core isolation do work on my Windows 11 guests, I suspect also because of lack of NV support in Xen. I also think over time this NV feature will become important also for other platforms that depend more on Linux than Windows does.
Has anyone here seen Windows 11 core isolation working on XCP-ng? One can check on a Windows 11 XCP-ng guest by looking at Windows Security -> Device Security -> Core Isolation -> Core Isolation Details in the Windows guest. I bet in every case it reports that it does not work. When I try to enable it, it successfully enables it and notifies me I need to reboot for the new setting to take effect, but when I reboot the core isolation feature is disabled again.
Apparently Windows virtualization, while important, is not important enough for deep-pocketed customers to push for this feature in XCP-ng and Xen upstream. The question I raise is, can a group of XCP-ng users, perhaps working in their home labs, get the ball rolling in upstream Xen without deep-pocket customers asking them to add the NV feature to Xen? I am hoping yes, because I think the Xen upstream developers really want to add this feature (Vates too, because it is a big negative for XCP-ng compared to other options that do support these Windows features that depend on NV).
But the Xen developers do not have the time to work on NV without deep-pocket customers asking for the feature. We can greatly improve the probability that the Xen developers will work on NV if we can do some of the work for them. I think there are some things we can do to help the Xen developers support NV. This is what I am proposing.
-
Chuckz said:
For example, important security features in Windows 11 such as core isolation do work on my Windows 11 guests...Edit: That is a typo. I meant to say that core isolation does not work on my Windows 11 guests, and I suspect it is because of the lack of the NV feature in Xen and XCP-ng.
So my point is that over time, you can forget about running Windows on any other hypervisor except Hyper-V if it is true that we can never use NV in production.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login