So I read about the 'VM snapshot with disk exclusion' feature.

Is this supported at the XAPI level now? If so, how would one go about using it?

Regards

@Danp these hosts are running:

Version 8.2.1 ()
Build number release/yangtze/master/58

I've been digging around vGPU for days.
I found information that you don't need any additional binaries to make it work. All you need is Nvidia vGPU drivers.
So I tried installing different versions of Nvidia vGPU drivers for XenServer on xcp-ng.
There were no errors during installation.
After installing the drivers I was able to see all vGPU types in XenCenter, nvidia-smi gives me the correct output.
I also checked xensource.log and this is what I found.

Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_gpumon] assert_vgpu_pgpu_are_compatible: vGPU/pGPU are compatible by default OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511/OpaqueRef:7e239728-ac74-4323-b8d0-7c40fa318411 Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_gpumon] assert_vgpu_pgpu_are_compatible: vGPU/pGPU are compatible by default OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511/OpaqueRef:6448a86a-ee72-492a-b700-83d1645f0c60 Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_gpumon] assert_vgpu_pgpu_are_compatible: vGPU/pGPU are compatible by default OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511/OpaqueRef:7e239728-ac74-4323-b8d0-7c40fa318411 Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_gpumon] assert_vgpu_pgpu_are_compatible: vGPU/pGPU are compatible by default OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511/OpaqueRef:6448a86a-ee72-492a-b700-83d1645f0c60 Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|vgpuops] vGPUs allocated to VM (OpaqueRef:7051bb05-712e-4dc3-bf6b-fef76c0980ee) are: OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511 Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|vgpuops] Creating virtual VGPUs Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_gpumon] assert_vgpu_pgpu_are_compatible: vGPU/pGPU are compatible by default OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511/OpaqueRef:7e239728-ac74-4323-b8d0-7c40fa318411 Dec 1 00:53:25 XEN60 xapi: [debug||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_gpumon] assert_vgpu_pgpu_are_compatible: vGPU/pGPU are compatible by default OpaqueRef:f8b54a1f-8f3c-4ba5-a475-980b4e2af511/OpaqueRef:6448a86a-ee72-492a-b700-83d1645f0c60 Dec 1 00:53:25 XEN60 xapi: [ info||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xenops] xenops: VM.import_metadata {"vusbs":[],"vgpus":[{"implementation":["Nvidia",{"extra_args":"","uuid":"a50159e3-a755-0b90-19ea-d5697b007834","type_id":"224","virtual_pci_address":{"fn":0,"dev":11,"bus":0,"domain":0}}],"physical_pci_address":{"fn":0,"dev":0,"bus":193,"domain":0},"position":0,"id":["f565e3ab-2fc9-2d00-a184-e7f28ee91915","0"]}],"pcis":[],"vifs":[],"vbds":[{"persistent":true,"extra_private_keys":{},"extra_backend_keys":{"polling-duration":"1000","polling-idle-threshold":"50"},"unpluggable":true,"ty":"CDROM","mode":"ReadOnly","position":["Ide",3,0],"id":["f565e3ab-2fc9-2d00-a184-e7f28ee91915","xvdd"]},{"persistent":true,"extra_private_keys":{},"extra_backend_keys":{"polling-duration":"1000","polling-idle-threshold":"50"},"unpluggable":true,"ty":"Disk","backend":["VDI","b2d22c67-abe8-d411-ba19-b5aa046407e9/b3fc6e5b-7d44-4117-b614-a9c086be0cf7"],"mode":"ReadWrite","position":["Ide",0,0],"id":["f565e3ab-2fc9-2d00-a184-e7f28ee91915","xvda"]}],"vm":{"generation_id":"4573398145280631014:2680438277400141975","has_vendor_device":true,"pci_power_mgmt":false,"pci_msitranslate":false,"on_reboot":["Start"],"on_shutdown":["Shutdown"],"on_crash":["Start"],"scheduler_params":{"affinity":[],"priority":[256,0]},"vcpus":2,"vcpu_max":2,"memory_dynamic_min":2147483648,"memory_dynamic_max":2147483648,"memory_static_max":2147483648,"suppress_spurious_page_faults":false,"ty":["HVM",{"firmware":["Uefi",{"backend":"xapidb","on_boot":"Persist"}],"qemu_stubdom":false,"qemu_disk_cmdline":false,"boot_order":"cd","pci_passthrough":false,"pci_emulations":[],"serial":"pty","acpi":true,"video":"Vgpu","video_mib":16,"timeoffset":"0","shadow_multiplier":1.0,"hap":true}],"bios_strings":{"bios-vendor":"Xen","bios-version":"","system-manufacturer":"Xen","system-product-name":"HVM domU","system-version":"","system-serial-number":"","baseboard-manufacturer":"","baseboard-product-name":"","baseboard-version":"","baseboard-serial-number":"","baseboard-asset-tag":"","baseboard-location-in-chassis":"","enclosure-asset-tag":"","hp-rombios":"","oem-1":"Xen","oem-2":"MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d"},"platformdata":{"featureset":"178bfbff-f6d83203-2e500800-040001f7-0000000f-219c01a9-00400004-00000000-010cd005-00000000-00000000-00000000-00000000-00000000-00000000-00000000-00000000-00000000","timeoffset":"0","usb":"true","usb_tablet":"true","device-model":"qemu-upstream-uefi","videoram":"8","hpet":"true","secureboot":"false","viridian_apic_assist":"true","apic":"true","device_id":"0002","cores-per-socket":"2","viridian_crash_ctl":"true","pae":"true","vga":"std","nx":"true","viridian_time_ref_count":"true","viridian_stimer":"true","viridian":"true","acpi":"1","viridian_reference_tsc":"true"},"xsdata":{"vm-data/mmio-hole-size":"268435456","vm-data":""},"ssidref":0,"name":"Windows Server 2022 (64-bit) (1)","id":"f565e3ab-2fc9-2d00-a184-e7f28ee91915"}} Dec 1 00:53:25 XEN60 xenopsd-xc: [debug||140 |Async.VM.start R:021195d15a31|xenops_utils] TypedTable: Writing VM/f565e3ab-2fc9-2d00-a184-e7f28ee91915/vgpu.0 Dec 1 00:53:25 XEN60 xenopsd-xc: [debug||36 |Async.VM.start R:021195d15a31|xenguesthelper] connect: args = [ -mode hvm_build -image /usr/libexec/xen/boot/hvmloader -vgpu -domid 2 -store_port 3 -store_domid 0 -console_port 4 -console_domid 0 -mem_max_mib 2032 -mem_start_mib 2032 ] Dec 1 00:53:26 XEN60 xenopsd-xc: [debug||36 ||xenops] Device.Dm.start domid=2 args: [-vgpu -videoram 16 -vnc unix:/var/run/xen/vnc-2,lock-key-sync=off -acpi -monitor null -pidfile /var/run/xen/qemu-dm-2.pid -xen-domid 2 -m size=2032 -boot order=cd -usb -device usb-tablet,port=2 -smp 2,maxcpus=2 -serial pty -display none -nodefaults -trace enable=xen_platform_log -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -S -parallel null -qmp unix:/var/run/xen/qmp-libxl-2,server,nowait -qmp unix:/var/run/xen/qmp-event-2,server,nowait -device xen-platform,addr=3,device-id=0x0002 -drive file=,if=none,id=ide1-cd1,read-only=on -device ide-cd,drive=ide1-cd1,bus=ide.1,unit=1 -device nvme,serial=nvme0,id=nvme0,addr=7 -drive id=disk0,if=none,file=/dev/sm/backend/b2d22c67-abe8-d411-ba19-b5aa046407e9/b3fc6e5b-7d44-4117-b614-a9c086be0cf7,media=disk,auto-read-only=off,format=raw -device nvme-ns,drive=disk0,bus=nvme0,nsid=1 -device xen-pvdevice,device-id=0xc000,addr=6 -net none] Dec 1 00:53:26 XEN60 xenopsd-xc: [debug||36 ||xenops] Starting daemon: /usr/bin/vgpu with args [--domain=2; --vcpus=2; --suspend=/var/lib/xen/demu-save.2; --device=0000:c1:00.0,224,0000:00:0b.0,a50159e3-a755-0b90-19ea-d5697b007834] Dec 1 00:53:26 XEN60 xenopsd-xc: [debug||36 ||xenops] vgpu: should be running in the background (stdout -> syslog); (fd,pid) = (FEFork (35,8823)) Dec 1 00:53:26 XEN60 xenopsd-xc: [debug||36 ||xenops] Daemon started: vgpu-2 Dec 1 00:53:56 XEN60 xenopsd-xc: [error||36 ||xenops] vgpu: unexpected exit with code: 127 Dec 1 00:53:56 XEN60 xenopsd-xc: [ info||36 ||xenops_server] Caught Xenops_interface.Xenopsd_error([S(Failed_to_start_emulator);[S(f565e3ab-2fc9-2d00-a184-e7f28ee91915);S(vgpu);S(Daemon exited unexpectedly)]]) executing ["VM_start",["f565e3ab-2fc9-2d00-a184-e7f28ee91915",false]]: triggering cleanup actions Dec 1 00:53:58 XEN60 xenopsd-xc: [error||36 ||task_server] Task 42 failed; Xenops_interface.Xenopsd_error([S(Failed_to_start_emulator);[S(f565e3ab-2fc9-2d00-a184-e7f28ee91915);S(vgpu);S(Daemon exited unexpectedly)]]) Dec 1 00:53:58 XEN60 xenopsd-xc: [debug||36 ||xenops_server] TASK.signal 42 = ["Failed",["Failed_to_start_emulator",["f565e3ab-2fc9-2d00-a184-e7f28ee91915","vgpu","Daemon exited unexpectedly"]]] Dec 1 00:53:58 XEN60 xapi: [ info||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xapi_network] Caught Xenops_interface.Xenopsd_error([S(Failed_to_start_emulator);[S(f565e3ab-2fc9-2d00-a184-e7f28ee91915);S(vgpu);S(Daemon exited unexpectedly)]]): detaching networks Dec 1 00:53:58 XEN60 xapi: [error||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xenops] Caught exception starting VM: Xenops_interface.Xenopsd_error([S(Failed_to_start_emulator);[S(f565e3ab-2fc9-2d00-a184-e7f28ee91915);S(vgpu);S(Daemon exited unexpectedly)]]) Dec 1 00:53:58 XEN60 xenopsd-xc: [debug||25 |org.xen.xapi.xenops.classic events D:22e2807de46a|xenops_utils] TypedTable: Removing VM/f565e3ab-2fc9-2d00-a184-e7f28ee91915/vgpu.0 Dec 1 00:53:58 XEN60 xenopsd-xc: [debug||25 |org.xen.xapi.xenops.classic events D:22e2807de46a|xenops_utils] TypedTable: Deleting VM/f565e3ab-2fc9-2d00-a184-e7f28ee91915/vgpu.0 Dec 1 00:53:58 XEN60 xenopsd-xc: [debug||25 |org.xen.xapi.xenops.classic events D:22e2807de46a|xenops_utils] DB.delete /var/run/nonpersistent/xenopsd/classic/VM/f565e3ab-2fc9-2d00-a184-e7f28ee91915/vgpu.0 Dec 1 00:53:58 XEN60 xapi: [error||952 HTTPS 192.168.8.103->|Async.VM.start R:021195d15a31|xenops] Re-raising as FAILED_TO_START_EMULATOR [ OpaqueRef:7051bb05-712e-4dc3-bf6b-fef76c0980ee; vgpu; Daemon exited unexpectedly ] Dec 1 00:53:58 XEN60 xapi: [error||952 ||backtrace] Async.VM.start R:021195d15a31 failed with exception Server_error(FAILED_TO_START_EMULATOR, [ OpaqueRef:7051bb05-712e-4dc3-bf6b-fef76c0980ee; vgpu; Daemon exited unexpectedly ]) Dec 1 00:53:58 XEN60 xapi: [error||952 ||backtrace] Raised Server_error(FAILED_TO_START_EMULATOR, [ OpaqueRef:7051bb05-712e-4dc3-bf6b-fef76c0980ee; vgpu; Daemon exited unexpectedly ])

I think that main is

Dec 1 00:53:26 XEN60 xenopsd-xc: [debug||36 ||xenops] vgpu: should be running in the background (stdout -> syslog); (fd,pid) = (FEFork (35,8823)) Dec 1 00:53:26 XEN60 xenopsd-xc: [debug||36 ||xenops] Daemon started: vgpu-2 Dec 1 00:53:56 XEN60 xenopsd-xc: [error||36 ||xenops] vgpu: unexpected exit with code: 127

So it said that Daemon started: vgpu-2, but then failed with code: 127

Are there any options to debug it? Any ideas why it failed to start?

@bullerwins It may be necessary to upgrade multiple times the proxy if the VM is not known to the main XO.

If it does not work, open a support tunnel on your XO and we'll take a look directly.

@Andrew thanks for the hint. Didn't even think of -or look much- into the shell commands, but you're absolutely right, they would be faster and easier than getting a XCP-ng Center running on some virtual Windows, when I can simply open the shell (even from within XOA) and do it this way.

@olivierlambert ,
Thank you. Someone from our team will reach out to you.

I think @florent did various modifications recently to handle those cases automatically.

Cloud configs allow {{name}}% substitutions. From my perspective (as a user), for how {{name}} is used in my cloud configs something like {{address}} would be identical.

What is the intended use case for the IPAM functionality in XOA if it can't be used like this? Real question, not trolling 🙂

@MrXeon So, the actual root issue here I believe, is opnsense installs come with an IP and dhcp server already assigned and enabled on the lan interface (I believe it's 192.168.1.1, but don't quote me). If your existing home network already uses 192.168.1.x/24 and already has a dhcp server, booting an opnsense install with it's virtual lan nic set to your existing home lan, there will be a lot of conflicts. Virtual nic order can be whatever you'd like (you can change and move around assignments in opnsense), but if it's preconfigured lan interface gets set to your preexisting lan network, there will be conflicts 🙂

Also note the text at the top of your screenshot: to continue you need to select a boot device. There might be a way in that menu (or partition creation submenu) to mark that created partition as bootable, or maybe you just need to highlight/select the partition under "used devices" before hitting "done"

You are welcome 🙂

I would like some examples too.

My latest (first) try gives me:

alfcruz-adm@local.hcpa.ufrgs.br@volt-ampere:~/packer-plugin-xenserver$ packer build ./ubuntu-2004.pkr.hcl xenserver-iso.ubuntu-2004: output will be in this color. Build 'xenserver-iso.ubuntu-2004' errored after 8 seconds 155 milliseconds: API Error: SESSION_AUTHENTICATION_FAILED root Authentication failure ==> Wait completed after 8 seconds 155 milliseconds ==> Some builds didn't complete successfully and had errors: --> xenserver-iso.ubuntu-2004: API Error: SESSION_AUTHENTICATION_FAILED root Authentication failure ==> Builds finished but no artifacts were created. alfcruz-adm@local.hcpa.ufrgs.br@volt-ampere:~/packer-plugin-xenserver$

Using root user. This user/pass works over SSH.

@Danp And of course if there is a real risk/issue, keep us posted and we'll take care of it 🙂

@isener FYI you can register the xo-cli with the SSH account of the VM and if you don't have an SSH account you can create one.
Follow the doc here: https://xen-orchestra.com/docs/troubleshooting.html#set-or-recover-xoa-vm-password

@encryptblockr hope to try the same with their even smaller "CORE" images. There is not an .OVA download but .xz downloads that when expanded have .img .

https://cdimage.ubuntu.com/ubuntu-core/22/stable/current/ubuntu-core-22-amd64.img.xz

and

https://cdimage.ubuntu.com/ubuntu-core/22/stable/current/ubuntu-core-22-amd64+intel-iot.img.xz

cloud-init seems to require experimentation and these small images boot quickly for me.

Are you then following Fohdeesha's instructions?

@olivierlambert I just saw Tom post about that. Already investigating about it. Thanks!
https://infosec.exchange/@tomlawrence/109404324178466606

@gnpcx What did you end up doing for this? The docs dont seem to have any guidance.

@lawrencesystems
thank you tom! really appreciate this!
a5ccebe1-49cb-4d23-9279-a2fda336c80f-image.png

seems that it is running without issues!

nice, thanks!

It is just relative no value per se