<![CDATA[OmniOS / Illumos / Solaris as a guest - not working]]>OmniOS is used by many people as storage solution with ZFS and passthrough devices (SSD&HDD) for NFS and SMB sharing. We have been using it for years on ESXI. Now because of ESXI Exodus, we have tried to set it up on XCP-ng.

OmniOS guest fails to boot in XCP-ng on version 8.2, 8.3.

Same goes for Illumos and Solaris 11.4.

Installation of OmniOS is fine: create a new vm, set viridian=false and installation progress smoothly.

When booting from freshly installed disk it panics with:
WARNING: Couldn't initialize xpv framework

panic[cpu0]/thread=ffffffffffbca07e0: unable to configure /xpvd nexus

a845404f-702e-41a2-ba19-0cdd1bc6f131-image.png

I've tried several version of OmniOS (and latest OpenIndiana), I also tried Solaris 11.4, tried disabling apix with apix_enable=0 in /etc/system and all without success.
The same setup in xcp-ng 8.0 works like a charm with working PV-drivers (but no tools).

The interesting thing is that there is an OmniOS image for AMAZON AWS - https://omnios.org/setup/aws which runs on XEN (or did they go to KVM?) so it should actually run? How can I get it to run... it seems very close.

Anyone with an idea of a workaround or even better, a resolution to this.

]]>https://xcp-ng.org/forum/topic/8371/omnios-illumos-solaris-as-a-guest-not-workingRSS for NodeSun, 11 Oct 2026 20:31:56 GMTTue, 13 Feb 2024 20:08:13 GMT60<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Fri, 06 Dec 2024 07:19:15 GMT]]>@TeddyAstie thank you for investigating.

]]>https://xcp-ng.org/forum/post/86568https://xcp-ng.org/forum/post/86568Fri, 06 Dec 2024 07:19:15 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Thu, 05 Dec 2024 14:50:17 GMT]]>@Buckle8711 Hello,

I took a look but unfortunately, I haven't found a workaround for this.
I suspect the Illumos/OmniOS Xen PV drivers to be too old to work in current Xen, but we currently don't have a way to prevent those drivers from loading (e.g HVM-only (non-PVHVM) domains).

]]>https://xcp-ng.org/forum/post/86538https://xcp-ng.org/forum/post/86538Thu, 05 Dec 2024 14:50:17 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Mon, 02 Dec 2024 12:54:17 GMT]]>@TeddyAstie thanks for looking into this. Do you have any updates here? Any workarounds or bug reports I can follow for updates?

]]>https://xcp-ng.org/forum/post/86416https://xcp-ng.org/forum/post/86416Mon, 02 Dec 2024 12:54:17 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Mon, 18 Nov 2024 12:06:35 GMT]]>@Teddy-Astie Yes...
xcp-oi-interfaces.jpg

]]>https://xcp-ng.org/forum/post/85941https://xcp-ng.org/forum/post/85941Mon, 18 Nov 2024 12:06:35 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Mon, 18 Nov 2024 09:04:51 GMT]]>@Andrew said in OmniOS / Illumos / Solaris as a guest - not working:

@olivierlambert XCP/Xen/Qemu expert question: Why does qemu start with only two interfaces when more are configured in XO?
qemu-dm-7 -machine pc-i440fx-2.10,accel=xen ... -device e1000,netdev=tapnet1,mac=5a:7f:c3:ba:04:d6,addr=5,rombar=0 -netdev tap,id=tapnet1,fd=7 -device e1000,netdev=tapnet0,mac=8e:68:02:55:da:c7,addr=4,rombar=0 -netdev tap,id=tapnet0,fd=8 ...
Where's the third and fourth? I see other VMs with more. Is it a template or vm-param issue?

These are the same "interfaces" just plugged differently to the guest (e.g tapnet0 exposed as e1000 and "tap" and tapnet1 exposed as e1000 and "tap" too).

You have 4 interfaces configured ?

]]>https://xcp-ng.org/forum/post/85935https://xcp-ng.org/forum/post/85935Mon, 18 Nov 2024 09:04:51 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Sat, 16 Nov 2024 23:01:50 GMT]]>Ping @Teddy-Astie if you know the best person internally to take a look at this

]]>https://xcp-ng.org/forum/post/85902https://xcp-ng.org/forum/post/85902Sat, 16 Nov 2024 23:01:50 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Sat, 16 Nov 2024 01:00:30 GMT]]>@Buckle8711 I see that too. QEMU is only started with two interfaces passed to the guest. I don't know if that's because of the lack of tools or native drivers or some other config issue.

@olivierlambert XCP/Xen/Qemu expert question: Why does qemu start with only two interfaces when more are configured in XO?
qemu-dm-7 -machine pc-i440fx-2.10,accel=xen ... -device e1000,netdev=tapnet1,mac=5a:7f:c3:ba:04:d6,addr=5,rombar=0 -netdev tap,id=tapnet1,fd=7 -device e1000,netdev=tapnet0,mac=8e:68:02:55:da:c7,addr=4,rombar=0 -netdev tap,id=tapnet0,fd=8 ...
Where's the third and fourth? I see other VMs with more. Is it a template or vm-param issue?

]]>https://xcp-ng.org/forum/post/85878https://xcp-ng.org/forum/post/85878Sat, 16 Nov 2024 01:00:30 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Fri, 15 Nov 2024 22:33:37 GMT]]>I've hit a blocker in my migration to xcp-ng. I'm using OmniOS as a lightweight file server with Napp-it management interface.

I reached a point where I needed to add two additional VIF's to support some additional VLANs bring the OmniOS VM total to four. Unfortunately only the first two are mapped to devices in OmniOS. Open to any ideas here.

]]>https://xcp-ng.org/forum/post/85875https://xcp-ng.org/forum/post/85875Fri, 15 Nov 2024 22:33:37 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Tue, 12 Nov 2024 05:18:20 GMT]]>@Buckle8711 No guest tools.... OpenSolaris does include Xen drivers but I don't know if they have been updated. I don't seem them loading.

]]>https://xcp-ng.org/forum/post/85633https://xcp-ng.org/forum/post/85633Tue, 12 Nov 2024 05:18:20 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Tue, 12 Nov 2024 02:23:25 GMT]]>@Andrew Thanks for confirming, it seems to be working fine so at least I can progress.
Can I ask what vm guest tools you are running? I dont see Solaris/Illumos/OmniOS listed in the wiki so not sure how to proceed.

]]>https://xcp-ng.org/forum/post/85630https://xcp-ng.org/forum/post/85630Tue, 12 Nov 2024 02:23:25 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Tue, 12 Nov 2024 01:25:41 GMT]]>@Buckle8711 It boots just fine for me with XCP 8.3... with the same PCI Warning... And the notices are not errors, so it works (I don't have the notices on my VM).

]]>https://xcp-ng.org/forum/post/85629https://xcp-ng.org/forum/post/85629Tue, 12 Nov 2024 01:25:41 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Tue, 12 Nov 2024 00:25:34 GMT]]>SCR-20241111-oloe.png Enabling 'Viridian' enables the client to boot but with some errors.

]]>https://xcp-ng.org/forum/post/85628https://xcp-ng.org/forum/post/85628Tue, 12 Nov 2024 00:25:34 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Tue, 12 Nov 2024 00:23:16 GMT]]>Another potential new user here looking to migrate from VMWare and evaluating Proxmox and xcp-ng.
I was leaning towards xcp-ng until I hit a blocker with laetst OmniOS 151052 and xcp-ng 8.3 that fails to boot in UEFI or BIOS modes. I tried 8.2 as well that also fails.

any suggestions greatly appreciated.

SCR-20241111-ojme.png

]]>https://xcp-ng.org/forum/post/85627https://xcp-ng.org/forum/post/85627Tue, 12 Nov 2024 00:23:16 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Mon, 29 Apr 2024 03:59:56 GMT]]>@mchadley002 @BobTB12 Thanks for the info! I concur, it boots now! (XCP 8.2 and 8.3)

Two important hints: First ENABLE Viridian for the VM. It does not want to boot without it. Second BOOT with UEFI, BIOS does work but disk IO sucks.

I used the ISO boot image: OI-hipster-minimal-20240426.iso

]]>https://xcp-ng.org/forum/post/76363https://xcp-ng.org/forum/post/76363Mon, 29 Apr 2024 03:59:56 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Mon, 29 Apr 2024 02:13:58 GMT]]>@BobTB12,

For what it's worth, I came back to a project I was working on trying to use OpenIndiana in a VM on XCP-ng 8.2 today and serendipitously, that project just released a new version 2024.04 on the same day. I gave the new ISOs a try and was able to boot into the installer and install it and then reboot into OI no problem. I tried both the GUI and text installers without an issue. I don't actually know what the difference is, but the release note does say "New Hardware Support" so I guess whatever that is worked! Not sure about other Illumos based distro; whether this is something later versions of them will pickup as well from their common base or if it's something that OI did specifically 🤷.

]]>https://xcp-ng.org/forum/post/76361https://xcp-ng.org/forum/post/76361Mon, 29 Apr 2024 02:13:58 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Thu, 15 Feb 2024 15:03:36 GMT]]>@Andrew
You gave me the idea to boot it in UEFI mode. This loads much further down, almost finishes compared to BIOS. Might be that the guest does not support 1.3 NVMe.

Is there a way in XCP-ng to change the controller/disk type to SCSI or SATA or something like that. In ESXI it is easy to do, I can not find this option here. This might then work!

nvme0: NVMe spec version 1.3
panic[cpu1]/thread=fffffe00233cdc20: programming error: invalid NS/format in cmd fffffe16dbc1dd40

Warning - stack not written to the dump buffer
fffffe00233cd710 genunix:dev_err+81 ()
fffffe00233cd750 nvme:nvme_check_generic_cmd_status+236 ()
fffffe00233cd7d0 nvme:nvme_set_features+e5 ()
fffffe00233cd800 nvme:nvme_write_cache_set+47 ()
fffffe00233cd8a0 nvme:nvme_init+9f0 ()
fffffe00233cd970 nvme:nvme_attach+51b ()
fffffe00233cd9e0 genunix:devi_attach+b1 ()
fffffe00233cda20 genunix:attach_node+8b ()
fffffe00233cda70 genunix:i_ndi_config_node+118 ()
fffffe00233cdaa0 genunix:i_ddi_attachchild+3e ()
fffffe00233cdae0 genunix:devi_attach_node+64 ()
fffffe00233cdb50 genunix:config_immediate_children+b0 ()
fffffe00233cdba0 genunix:devi_config_common+6a ()
fffffe00233cdc00 genunix:mt_config_thread+10b ()
fffffe00233cdc10 unix:thread_start+b ()
]]>https://xcp-ng.org/forum/post/71371https://xcp-ng.org/forum/post/71371Thu, 15 Feb 2024 15:03:36 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 21:09:04 GMT]]>@BobTB12 On XCP 8.3/Xen 4.17 the OI installer boots and runs but errors out (not kernel crash) before it finishes. The installed OS is bootable but has the errors and does not finish booting. This is with BIOS boot as UEFI boot fails.

]]>https://xcp-ng.org/forum/post/71328https://xcp-ng.org/forum/post/71328Wed, 14 Feb 2024 21:09:04 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 20:31:10 GMT]]>@Danp Just now tried on Xen 4.17. Still the same. Seems as @Andrew said, on older XCP it did still work.

]]>https://xcp-ng.org/forum/post/71326https://xcp-ng.org/forum/post/71326Wed, 14 Feb 2024 20:31:10 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 17:04:19 GMT]]>@BobTB12 My guess would be the guest, but it would be interesting to know if the behavior changes on a newer version of Xen.

]]>https://xcp-ng.org/forum/post/71312https://xcp-ng.org/forum/post/71312Wed, 14 Feb 2024 17:04:19 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 16:17:39 GMT]]>@Danp Do you think this is hypervisor problem or guest problem?

]]>https://xcp-ng.org/forum/post/71309https://xcp-ng.org/forum/post/71309Wed, 14 Feb 2024 16:17:39 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 14:10:33 GMT]]>@Andrew
Wow, interesting... I am installing this on 8.3 beta 4 (the latest). I thought it is broken in the new 8.3, but is broken for 8.2.1 too...

I tried Solaris 11.4 again (the free one) and I could perhaps make it work if I could enter the console at boot to set grub. The console just ignores my keystrokes. I am now completely out.

]]>https://xcp-ng.org/forum/post/71299https://xcp-ng.org/forum/post/71299Wed, 14 Feb 2024 14:10:33 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 13:41:22 GMT]]>@BobTB12 Very odd.... A while ago I installed OI Hipster and booted it as a test. I just booted the installed VM and it works. I tried installing it again and I can't get it to install/boot now but the old install still works.

It looks like some Xen change in XCP 8.2.1, but I don't know what or when...

The good news, I just installed it on XCP 8.3 and it works..... but it's slow because it does not have any Xen drivers.

]]>https://xcp-ng.org/forum/post/71298https://xcp-ng.org/forum/post/71298Wed, 14 Feb 2024 13:41:22 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Wed, 14 Feb 2024 09:40:17 GMT]]>@DustinB
Yep, this is the problem we are having.

Interestingly it works in ESXI and Proxmox. Some simple thing maybe in the XCP-ng setting to make it work would be a dream 🙂

xpvd is Xen Paravirtualized Driver?

It seems its not the disk controller driver problem, as the setup finishes ok. Some device is being started thereafter (svga? network?) which causes the crash.

]]>https://xcp-ng.org/forum/post/71273https://xcp-ng.org/forum/post/71273Wed, 14 Feb 2024 09:40:17 GMT<![CDATA[Reply to OmniOS / Illumos / Solaris as a guest - not working on Tue, 13 Feb 2024 22:00:35 GMT]]>Well dang it, I could've swore it booted once.. but now I'm getting

04f70afa-0084-406d-ab21-1e39a79abdad-image.png

]]>https://xcp-ng.org/forum/post/71266https://xcp-ng.org/forum/post/71266Tue, 13 Feb 2024 22:00:35 GMT