Extracting data from XO with terraformer

michael132

I've recently come across the terraformer project (https://github.com/GoogleCloudPlatform/terraformer/) which supports a number of providers including xenorchestra.
I am failing to get it to extract the metadata for a single VM to create the terraform code needed to rebuild the VM.
As I understand it, I need to specify a resource and a filter to get the VM I want.
When I run

terraformer import xenorchestra list

to list the supported resources, there is no mention of 'vm', just 'acl' and 'resource_set'.
I will be extremely grateful if someone can show me how to achieve this.

olivierlambert

Pinging @ddelnano

ddelnano

Hey @michael132, apologies for the late reply.

The GitHub issue (vatesfr/terraform-provider-xenorchestra#96) tracking its initial implementation has some additional details and why the VM resource was not done initially (there's outstanding work to make the experience seamless). Since it was initially developed, I haven't received any feedback on the integration so I wasn't sure if it was worthwhile to maintain it.

If you are working with a single VM, have you tried terraform import? From my own personal terraformer use, I'm mainly found it valuable once you are trying to export a significant number of resources.

michael132

Hi @ddelnano,
many thanks for the reply.
I don't know how our existing infrastructure compares to others but we have ~50 Hypervisors (all running 8.2.1) and ~2000 VMs.
The VMs are running a mixture of Debian and Ubuntu and we are planning to upgrade them to a LTS version of Ubuntu.
Most of the VMs are essentially appliances that don't hold state so we decided that it would be easier to rebuild them using terraform rather than upgrade them in situ.
I am currently exploring options for pulling the data out of XO but I was kind of hoping that somebody might have already done the heavy lifting

ddelnano

@michael132 sorry I read a little too much into your initial message's "I am failing to get it to extract the metadata for a single VM".

That makes sense. The terraformer integration could help here, but as mentioned above it will require some work to integrate with the VM resource. We also would need to upgrade the terraform-provider-xenochestra dependency since it's quite old.

In order to understand how much work is needed, it would be useful to follow the terraform import process for one of these VMs and see what difficulties are surfaced through that process. The terraform import process will share the same issue(s) that terraformer will see as mentioned in my comment here.

Identifying the outstanding work would help us understand if there is more to complete outside of terraform-provider-xenorchestra#100 and the terraformer glue work itself.

michael132

@ddelnano, what arguments do I need to pass to the 'terraform import' command?

ddelnano

@michael132 here is the command you will need to run, but I recommend doing some research on this command's workflow if you aren't familiar.

# Note: you will need to have a barebones resource
# specified that matches the later `terraform import` command
$ cat main.tf

resource "xenorchestra_vm" "vm1" {

}

# See screenshot below for the uuid example.
$ terraform import xenorchestra_vm.vm1 uuid-from-xo

From researching this, it seems the new import block can also generate terraform code. It may not script as well as terraformer would, but the functionality has some overlap. This blog explained the legacy import flow and the new import block and I recommend reading it.

michael132

@ddelnano I've been using terraform import to extract configuration data from xo.
When I run something like terraform import xenorchestra_vm.vm1 <UUID of actual VM> it completes successfully but when I run terraform plan it reports lots of errors.
The Missing required argument errors can be removed by adding dummy values to the resource eg (cpus = 0).
There are also Insufficient disk blocks and Insufficient network blocks errors.
When I examine the terraform.tfstate file, all that config is there (including the missing arguments).
I'm at a bit of a loss to understand wehat's (not) going on here.

Gurve

@michael132 First post so hopefully I don't mess up the formatting too bad.

I did the import with openTofu (AFAIK it's identical to terraform since it uses the terraform provider behind the scenes).

After import I got some fields during "plan" action that said it would need to be recreated
I followed the blogpost here:
https://xen-orchestra.com/blog/managing-existing-infrastructure-with-terraform-2/

the 2 offending fields for me was:

template
destroy_cloud_config_vdi_after_boot

so I added the small lifecycle part in the bottom and now "plan" is agreeing with the configured infrastructure.

I also added some math for memory and disk to make it easier to see how many GB it is easier.

resource "xenorchestra_vm" "imported" {
    template           = "template"
    auto_poweron       = true
    cpus               = 2
    exp_nested_hvm     = false
    hvm_boot_firmware  = "bios"
    memory_max         = 4*1073741824
    name_description   = "k8s-clu01-01"
    name_label         = "K8S-clu01-01"
    start_delay        = 0
    tags               = [
        "k8s-clu-01",
    ]
    cdrom {
        id = "some-cdrom-id"
    }

    disk {
        attached         = true
        name_description = "Created by XO"
        name_label       = "Debian Bookworm 12_evibo"
        size             = 50*1073741824
        sr_id            = "some-sr-id"
    }

    network {
        attached       = true
        mac_address    = "de:ad:be:ef"
        network_id     = "some-network-id"
    }
    lifecycle {
      ignore_changes = [
        template,
        destroy_cloud_config_vdi_after_boot,
      ]
    }
}

ddelnano

@Gurve thank you for providing a full example!