Actually, Xen never officially supported Nested Virtualization. It was experimental, and broke when other needed changes were made to Xen. Now there's work to be done to make it fully supported, and this won't happen before the final release of XCP-ng 8.3. This will be documented in the release notes.

This is also an issue for us internally as we create a lot of virtual pools for our tests.

I read through a lot of the earlier posts and finally started scrolling to find this, which is the answer I was looking for. Why do I care? There is a Microsoft evaluation learning lab for things like Intune that runs in Hyper-V, basically a bunch of VHD (x) that get spawned as needed. Applications I need to teach myself. Running XCP-NG 8.3 current updates for this lab.

If it doesn't happen, then I'll just need to throw an eval version of Windows Server on something else like an HP T740 to run these labs, not the biggest issue for me.

Link for the labs if anyone is curious (free with an email registration like all the evals):

https://www.microsoft.com/en-us/evalcenter/evaluate-mem-evaluation-lab-kit

I'd think direct Docker support would be a higher priority than nested virtualization with a focus on Hyper-V. But that's just me.

https://xcp-ng.org/forum/topic/8932/hardware-assisted-virtualization-is-not-enabled-on-this-host-even-though-platform-exp-nested-hvm-true-is-set/52

Nested Virtualization (X86) Part I - George Dunlap, Xen Server:
https://www.youtube.com/watch?v=8jKGYY1Bi_o

Nested Virtualization (X86) Part II - George Dunlap, Xen Server:
https://www.youtube.com/watch?v=3MxWvVTmY1s

Maybe the only option is going to be getting a new physical host and then using Hyper-V on that? Hate to say it, but sounds like it could be the final result.

I did that with ESXi for this one vendor since I didn't have a choice at the time, it was lame, but the company understood the need for the expense, though it helped that we were going to need another (albeit not as powerful) host anyway.

They have straight-up refused to discuss trying anything outside of Hyper-V and completely acted stupid when I mentioned something about moving the VHD that their software creates in Hyper-V and put it in our hypervisor directly... Said that's unsupported and we will not assist you.

Still trying to figure out how to get around this... I had Hyper-V running on a Proxmox VM but the software seems to stall out and never finish installing. I'm trying XCP-NG 8.2 but based on all the conversations here, I'm assuming Hyper-V will refuse to run.

I'm honestly a little surprised any vendors require Hyper-V too, like, if you're going to require something, why not use the industry standard that is ESXi?

And if that was the case, it would be an OVA which would be something you could natively import to most hypervisors anyway.

What an interesting situation lol.

I got it working well on a Hyper-V setup once, but the nested VM still had some odd issues, bad latency, and a few other things, and that was all Windows based stuff so it was kinda a best case scenario.

Good luck with the vendor, hopefully they can be convinced that it will work just fine on other hypervisors.

One vendor I worked with that required ESXi for their Windows VM finally changed their minds and worked w/ me to do some validation on XCP-ng. After we ran through a ton of testing (this was a very high bandwidth/data usage platform with strict requirements) the engineers were flabbergasted because XCP-ng performed so much better lol. I was like "you're really surprised something is faster than bloated ESXi??"

I'm experimenting with Nested virtualization with Proxmox... The application is installing but throwing errors, but I don't know if that's because of it being nested or if it's just throwing errors...

Regardless I may try XCP-NG 8.2 latest and see how it reacts with nested virtualization and just stick with that if it works... It'll be easier to backwards migrate everything from 8.3 to 8.2 than moving to Proxmox (I think?)

Still should never be done in a production setup though so I don't think there should be any rush ha!

Would be very cool to have in lab environments though.

@planedrop In general, the fact is that the current XCP-ng 8.3 RC1 and XenServer 8 simply do not support any kind of nested virtualization.

Indeed.

Actually, Xen never officially supported Nested Virtualization. It was experimental, and broke when other needed changes were made to Xen. Now there's work to be done to make it fully supported, and this won't happen before the final release of XCP-ng 8.3. This will be documented in the release notes.

This is also an issue for us internally as we create a lot of virtual pools for our tests.

I still agree that it would be nice to have 1 box that runs them all, for sure, but there aren't any that can actually do that.

P. S.: In more legacy Xen, which is part of XCP-ng 8.2, nested virtualization works.

P. P. S.: https://xcp-ng.org/forum/topic/8932/hardware-assisted-virtualization-is-not-enabled-on-this-host-even-though-platform-exp-nested-hvm-true-is-set

Good luck!

I may play around with nested Hyper-V here soon to see what I can get working, but honestly I'd still avoid it EVEN IF we can get it working stable, could be a headache to troubleshoot down the road.

But yeah I may take your advice and call the vendor.

I have had issues with nested virt in ProxMox as well, admittedly less than in XCP-ng, but it's still not good enough on either that I'd use it consistently.

Is there a reason the application needs Hyper-V? The reason I ask is, I've had something similar due to a vendor who required ESXi for their VMs. Turns out there was no good reason for that and we ended up having to spend a ton of money for no real benefit. Has created more headaches down the road too w/ lack of central backups, etc... and massive wasted resources (time, money, all of the above). The vendor has since said they'll help us move to XCP-ng, but it's a big process and there was no reason for ESXi in the first place.

So maybe, just maybe, there's a chance the vendor/application developer can be convinced otherwise? One of the points of virtualization is keeping options open.

I have a client that needs to use part of Hyper-V for an application they are using. Are there any workarounds? I see Proxmox seems to support this, but I haven't tested.

Thanks!