Vagrant causing XCP-ng VM (with Nested Virtualization ON) to abruptly reboot

itnok

Am experiencing a weird behavior and wondering if anybody experienced something similar or has some hints/guidance about it.

On a server running XCP-ng, VMs (running Ubuntu 18.04) with nested virtualization ON are rebooting abruptly when Vagrant (2.2.14) is trying to bring up a libvirt/kvm “sub-guest” machine (NO issue if the machine uses QEMU rather… although too slow for the purpose!). The last line logged by vagrant before the XCP-ng VM restarts itself are something like:

==> vagrant-5-project-2-concurrent-0-job-681: Creating shared folders metadata...
==> vagrant-5-project-2-concurrent-0-job-681: Starting domain.
==> vagrant-5-project-2-concurrent-0-job-681: Error when updating domain settings: undefined method 'strip' for nil:NilClass
==> vagrant-5-project-2-concurrent-0-job-681: Waiting for domain to get an IP address...
==> vagrant-5-project-2-concurrent-0-job-681: Waiting for SSH to become available...

The VM just blows up and it reboots. It was working fine until some time ago. It looks like an update of XCP-ng might have broken it?

Anyone experiencing similar problems?
Is anyone able to provide some advice?

Thank!

olivierlambert

Nested code in Xen is not ultra-stable (it's also sensitive to your hardware). So I'm not really surprised about this...

itnok

@olivierlambert sad to know! Most of all as it was working beautifully until very recently...

Thanks for the feedback though.

Do you have suggestion for CI using Vagrant to spin up machines hosted on XCP-ng? Is Vagrant xen-server plug-in considered robust enough? (Not judging from its latest version number... 0.0.6... double-O is license to kill, right?!)

Thanks!

olivierlambert

That's a good question. If I could, I would do a Vagrant plugin for XO (instead of XCP-ng).

I know @ddelnano started to work on a Packer plugin, but I don't know how hard it is to write a Vagrant plugin.

I never used Vagrant myself, but anyone interested to work on that is very welcome!

itnok

@olivierlambert was thinking about this specifically...

https://github.com/xapi-project/vagrant-xenserver/releases

It looks like it did not receive that much love recently, but it should work...

ddelnano

@itnok yea it appears the project uses the plugin api for Vagrant 1.x which may not work with Vagrant 2.x (the latest version) I'm not sure if the plugin api has changed significantly since I haven't written any plugins myself.

olivierlambert

@itnok I would prefer a Vagrant plugin for XO, not XCP-ng.

itnok

@olivierlambert probably dumb question... but where can I find a clear complete overall document listing all endpoints for the Xen Orchestra API?

Tried the Vagrant plug-in mentioned above but I do agree it would be better to have a version of it targeting XO API rather than directly Xen/XCP-ng XAPI...

olivierlambert

@itnok Good question Take a look there: https://xen-orchestra.com/docs/architecture.html#api and here for the list of all endpoints: https://xen-orchestra.com/docs/architecture.html#xo-cli-cli

digit

I have same issue with xcp-ng running eve-ng (network simulator). eve-ng VM crash when launching qemu with "accel=kvm".

netsted virt and e1000

xe vm-param-get uuid=08c5e1be-f1a6-2179-6d0f-1360f456e6e6 param-name=platform
nic_type: e1000; timeoffset: 1; exp-nested-hvm: true; secureboot: false; videoram: 8; hpet: true; device-model: qemu-upstream-compat; apic: true; device_id: 0001; vga: std; nx: true; pae: true; viridian: false; acpi: 1

eve-ng is built on ubuntu 18.04

root@eve-ng:~# cat /sys/module/kvm_intel/parameters/nested
Y
root@eve-ng:~# uname -a
Linux eve-ng 4.20.17.2-eve-ng-uksm-wg+ #1 SMP Sat Feb 6 11:21:53 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

AlexanderK

i am trying to setup kypo-lite.
It needs a ubuntu vm with nested virtualization enabled .
It uses vagrant to setup everything and i have the same problem.
Vms reboot while vagrant setting up everything.

How can i pinpoint xcp-ng logs to help troubleshooting it?

AlexanderK

@olivierlambert any info about this

olivierlambert

It's really hard to tell, I would suggest to ask directly at the Xen community. We have far more urgent things to deal with at the moment sadly