Posts made by mlustosa | XCP-ng and XO forum

mlustosa

@TITUS-MAXIMUS https://github.com/terra-farm/terraform-provider-xenorchestra/issues/243

killroy1971 created this issue in terra-farm/terraform-provider-xenorchestra

closed Can't create VM without specifying an affinity_host - issue with an empty string #243

mlustosa

@olivierlambert said in Support for XO in docker container:

Feedback is always welcome

In short, we want to stay agnostic on XO sources. You can build whatever you want around it.

It was all I needed to know right after my initial suggestion.

I already use XO from sources with Docker, I have no difficulty in that and i'm grateful for this fantastic tool to exist. As I said, everything I said earlier are suggestions for new users.

But I understand your point now.

mlustosa

@d1rtym0nk3y said in Support for XO in docker container:

Is there likely to be any official support for running XO in a containerised environment in the near future?

So, from there I made my contribution to the benefits of using XO as a container.

After that you came to talk about XOA and that it would be a feature that wouldn't be worth it (financially) to implement. My surprise was when I did that.

One question: Can I, as a community user, open a Pull Request in the official git repository containing Docker files (dockerfiles and docker-compose and scripts) to be used as a complementary installation to the traditional method via VM?

mlustosa

I am quite surprised by your comment.

I thought I was in a forum where we discussed since the implementation of XOA as well as XO (compiled from the sources). You may even want to differentiate the community from paying and non-paying, but it certainly hurts the collaboration principle of the project that I got involved in and I still try to contribute (XCP-ng and XO), as ex and still xenserver user.

I didn't talk about XOA users here, I just reported my experience with docker containers and how this option would work very well in as step-to-step in install from sources and not to replace the installation and easy management of XOA as a whole.

I know that everyone needs money and that enabling new features, fixing bugs and maintaining the tool is expensive in men/hour, but disregarding a community user who believes in this project (like me) for the simple fact of not paying brings the idea that you don't care and not listen to the community.

Again, is this forum only for XOA users? Where i can improvement proposals be suggests? Here on the forum or on the official git respository?

This recalled the end that XenServer (opensource) had...

mlustosa

@olivierlambert said in Support for XO in docker container:

The most popular container is https://hub.docker.com/r/ezka77/xen-orchestra-ce

(2.4M downloads)

But it's not officially supported and you don't have an updater and all XOA features

Just to clarify the issue of updates. The good practice of working with Docker recommends that you maintain an "immutable infrastructure" when uploading a container (with docker-compose.yaml or sudo docker run ...). This means that after the images are created, it does not make sense to keep changing the container, write inside it, or replace processes at the time of its execution. Containers are designed to be ephemeral microservices and must be treated as such (and not as a VM). Briefly, the steps to create and update a microservice are:

Build a local image;
Run the container (replacing environment variables at the time of execution, such as passwords and volume locations)
(an update has been made to the official xen ochestra repository here)
Recreate the previously generated image (step 1) keeping tags or not;
Run the container again from the new image;

That is, it is not interesting to keep altering or making major modifications to a container, because if the image is not updated with these modifications, if the container is deleted or lost, everything you will have done will be lost.

mlustosa

@olivierlambert Aren't we talking about XO? If so, a docker image compiled from sources will never have all the features of the XOA (paid appliance), correct?

mlustosa

I vote in favor of maintaining a docker version of Xen Orchestra. The advantage of using container instead of VM is the ease of migration and scalability of the microservice. You can also upload several services in a single VM (and securely isolate them all). It will save processing and you can even easily replicate these containers via Docker Swarm or Kubernetes.

I'm using my own image where I work and uploading modifications to the original repository, cited by the @d1rtym0nk3y.
I am managing two pools, totaling 50 VMs. My total processing with the containers:

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
de7b8a1e6979        xo_server           0.00%               170.5MiB / 9.754GiB   1.71%               11.4GB / 2.95GB     0B / 0B             21
535a11147093        xo_redis            0.33%               11.46MiB / 9.754GiB   0.11%               63.8MB / 224MB      0B / 0B             6

The VM is consuming only 400MB of RAM.

I also experimented with Xen Ochestra installations using an entire Debian VM and the initial installation alone consumed much more than that.

I also already commit to sending updates and contributing with what I've already done in my repository if the XCP-ng team (@olivierlambert and others...) creates a folder in the official repository or a new Xen Orchestra repository as a container.

mlustosa

@olivierlambert said in Cloudinit does not work correctly:

@mlustosa said in Cloudinit does not work correctly:

ssh-ed25519

This is probably the issue. Are you sure you added correctly your key manually?

it's correct. I have two keys, one id_ed25519 and the other id_rsa.

mlustosa

@olivierlambert
I found out where the error is. For some reason it doesn't load the user-data file (cloud-config) if the network-config has something written, even commented out (starting with #).

I removed the file out of the config drive and restarted the VM and then it worked! It managed to add the public key ssh and install some packages on the system. If in the act of creating the VM in the Xen Orchestra I leave the network-config blank also works.

Why doesn't he accept the commented network-config file?

As in the act of creation the file is commented (network-config), so I thought that keeping it that way would not influence anything.

mlustosa

/mnt/dados -> /dev/xvdb (config drive with SSH-key)
/mnt/dados2 -> /dev/xvdc (config drive with custom config)

[centos@localhost ~]$ blkid /dev/xvdb
/dev/xvdb: SEC_TYPE="msdos" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat"

[centos@localhost ~]$ blkid /dev/xvdc
/dev/xvdc: SEC_TYPE="msdos" LABEL="cidata" UUID="355A-4FC2" TYPE="vfat"

[centos@localhost ~]$ fdisk -l

Disk /dev/xvdb: 10 MB, 10485760 bytes, 20480 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System

Disk /dev/xvdc: 10 MB, 10485760 bytes, 20480 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System

[centos@localhost ~]$ diff /mnt/dados /mnt/dados2

diff /mnt/dados/meta-data /mnt/dados2/meta-data
1c1
< instance-id: 88d79afd-2970-176a-e9e9-916540d993c2
---
> instance-id: 9a9aad16-b51e-cd6c-4f6e-5afe9b757d4e
Only in /mnt/dados: network-config
Common subdirectories: /mnt/dados/openstack and /mnt/dados2/openstack
diff /mnt/dados/user-data /mnt/dados2/user-data
2c2
< hostname: centos7-xentools0
---
> hostname: CentOS-7-cloudinit-withxentools
4c4,5
<    - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM2eZQHBkRY7BUZ+sJG9JH9nngFOmednoMG6Gxn7Trrq local marllus
---
>   - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM2eZQHBkRY7BUZ+sJG9JH9nngFOmednoMG6Gxn7Trrq local marllus
>   - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDwklFtEWwv6frHY3RtzLzV4Cjk42Z6e+DoSSVo2Oc5BVRBUAgQ0TcjdJQKxQt5VxZs7inRlFcOJ/TU8PYveb0n7uPNDQRXHKKFFq/G6ByGORiMWsfGrA7CQe7lTLFmcDprqqxEuhfMNjC5wL9kegTS2boU+u8PyKFKnTT91ivPuaOCwhHM0k2N6Q+PIsuBj6UsrAt08XQHCwup2jvM+MrtQVRju+kHMhXHvUeOP19gCBkSFgU+jKJoHFQ14Fce4z9K+TvYqyBIlHJh/aTV9iTfsnbb3KHRCMu2hxD0V2TPCrlz24xajuKqQxAWieNbhcquRKPuU58jeegZ8VwzbaV3 marllus@peba

mlustosa

I took a new test. I saved the VDI created (config drive) by Xen Orchestra using the SSH-Key option. I created a VM with cloud-init installed and before loading it I connected this VDI to it. It worked, including the packages option, as shown below:

#cloud-config
hostname: centos-app
ssh_authorized_keys:
  - ssh-rsa <myKey>
packages:
  - vim
  - git
  - tcpdump

When the VM starts, the public key ssh is in ~/.ssh/authorized_keys and the vim, git and tcpdump packages are installed.

Which generates the following hypothesis: The problem is in the generation of the VDI config drive (with custom config) using Xen Orchestra.

Any direction?

mlustosa

@olivierlambert
I compared and as I said, really with SSH-key only he can read the data and with custom config not.

/var/log/cloud-init.log after applying SSH-key:

2020-07-16 11:56:48,074 - util.py[DEBUG]: Running command ['mount', '-o', 'ro,sync', '-t', 'auto', u'/dev/xvdb', '/run/cloud-init/tmp/tmpmejw9b'] with allowed return codes [0] (shell=False, capture=True)
2020-07-16 11:56:48,141 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpmejw9b//user-data (quiet=False)
2020-07-16 11:56:48,143 - util.py[DEBUG]: Read 475 bytes from /run/cloud-init/tmp/tmpmejw9b//user-data
2020-07-16 11:56:48,143 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpmejw9b//meta-data (quiet=False)
2020-07-16 11:56:48,144 - util.py[DEBUG]: Read 50 bytes from /run/cloud-init/tmp/tmpmejw9b//meta-data
2020-07-16 11:56:48,144 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpmejw9b//vendor-data (quiet=False)
2020-07-16 11:56:48,144 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmpmejw9b//network-config (quiet=False)
2020-07-16 11:56:48,144 - util.py[DEBUG]: Running command ['umount', '/run/cloud-init/tmp/tmpmejw9b'] with allowed return codes [0] (shell=False, capture=True)
2020-07-16 11:56:48,159 - util.py[DEBUG]: Attempting to load yaml from string of length 50 with allowed root types (<type 'dict'>,)
2020-07-16 11:56:48,160 - DataSourceNoCloud.py[DEBUG]: Using data from /dev/xvdb
2020-07-16 11:56:48,161 - util.py[DEBUG]: Reading from /etc/hosts (quiet=False)
2020-07-16 11:56:48,161 - util.py[DEBUG]: Read 158 bytes from /etc/hosts
2020-07-16 11:56:48,164 - handlers.py[DEBUG]: finish: init-local/search-NoCloud: SUCCESS: found local data from DataSourceNoCloud
2020-07-16 11:56:48,164 - stages.py[INFO]: Loaded datasource DataSourceNoCloud - DataSourceNoCloud [seed=/dev/xvdb][dsmode=net]

/var/log/cloud-init.log after applying custom config:

2020-07-15 22:12:06,723 - util.py[DEBUG]: Running command ['mount', '-o', 'ro,sync', '-t', 'auto', u'/dev/xvdb', '/run/cloud-init/tmp/tmplcFFJh'] with allowed return codes [0] (shell=False, capture=True)
2020-07-15 22:12:06,801 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//user-data (quiet=False)
2020-07-15 22:12:06,802 - util.py[DEBUG]: Read 232 bytes from /run/cloud-init/tmp/tmplcFFJh//user-data
2020-07-15 22:12:06,802 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//meta-data (quiet=False)
2020-07-15 22:12:06,803 - util.py[DEBUG]: Read 50 bytes from /run/cloud-init/tmp/tmplcFFJh//meta-data
2020-07-15 22:12:06,803 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//vendor-data (quiet=False)
2020-07-15 22:12:06,803 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//network-config (quiet=False)
2020-07-15 22:12:06,805 - util.py[DEBUG]: Read 104 bytes from /run/cloud-init/tmp/tmplcFFJh//network-config
2020-07-15 22:12:06,805 - util.py[DEBUG]: Running command ['umount', '/run/cloud-init/tmp/tmplcFFJh'] with allowed return codes [0] (shell=False, capture=True)
2020-07-15 22:12:06,825 - util.py[DEBUG]: Attempting to load yaml from string of length 50 with allowed root types (<type 'dict'>,)
2020-07-15 22:12:06,826 - util.py[DEBUG]: Attempting to load yaml from string of length 104 with allowed root types (<type 'dict'>,)
2020-07-15 22:12:06,827 - util.py[DEBUG]: loaded blob returned None, returning default.
2020-07-15 22:12:06,827 - handlers.py[DEBUG]: finish: init-local/search-NoCloud: FAIL: no local data found from DataSourceNoCloud
2020-07-15 22:12:06,827 - util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'> failed
2020-07-15 22:12:06,828 - util.py[DEBUG]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'> failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 743, in find_source
    if s.update_metadata([EventType.BOOT_NEW_INSTANCE]):
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 632, in update_metadata
    result = self.get_data()
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 256, in get_data
    return_value = self._get_data()
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceNoCloud.py", line 126, in _get_data
    mydata = _merge_new_seed(mydata, seeded)
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceNoCloud.py", line 353, in _merge_new_seed
    util.load_yaml(seeded.get('network-config')))
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceNoCloud.py", line 324, in _maybe_remove_top_network
    network_val = cfg.get('network', nullval)
AttributeError: 'NoneType' object has no attribute 'get'
2020-07-15 22:12:06,842 - main.py[DEBUG]: No local datasource found

In summary, in the first case I have:
Loaded datasource DataSourceNoCloud - DataSourceNoCloud [seed=/dev/xvdb][dsmode=net]

And in the second I have:
handlers.py[DEBUG]: finish: init-local/search-NoCloud: FAIL: no local data found from DataSourceNoCloud

Another question, according to this tutorial, besides the possibility of configuring the public key, the config drive (SSH-key) also changes the hostname to the name defined in the template, however, even with SSH-key enabled it changes the name for localhost (and not for the name fixed in the template).
Below is a demonstration after booting the VM with SSH-key cloud drive enabled.

[root@localhost cloud] cat /var/lib/cloud/data/set-hostname 
{
 "fqdn": "localhost", 
 "hostname": "centos-app"
}

[root@localhost cloud] hostname
localhost

mlustosa

I'm using Xenserver 7.1 and XenOrchestra (xo-server 5.62.0 and xo-web 5.63.0).
I'm following this tutorial and installed a CentOS 7.0 minimal to create a cloudinit template from scratch.
When trying to upload a VM from the created template, it can only load the cloud drive if I use the SSH-Key option (the first option below).
alt text

However, if I put the same configuration in the custom config option it doesn't work. Below is the configuration I am using (replacing with my corresponding key in ssh-rsa <myKey>). for example, following this tutorial.

# cloud-config
hostname: proxy
ssh_authorized_keys:
   - ssh-rsa <myKey>

When I try to connect via ssh to the newly created VM, it gives a permission error.

ssh -i ~/.ssh/mykey centos@10.xx.xx.109

so...

Warning: Permanently added '10.xx.xx.109 '(ECDSA) to the list of known hosts.
centos@10.xx.xx.109: Permission denied (publickey, gssapi-keyex, gssapi-with-mic).

Right after that I connect to the VM via root and go to /home/centos/.ssh/authorized_keys and there's nothing inside.
I tested it by also putting parameters for changing the hostname and installing some packages in the custom config and none of them worked for me.

I looked at /var/log/cloud-init to check for something strange and saw nothing that could indicate an error. He can even mount the disk containing the settings (config drive) /dev/xvdb. Here is a part of a part of the attempt to connect to the config drive and read the files.

2020-07-15 22:12:06,723 - util.py[DEBUG]: Running command ['mount', '-o', 'ro,sync', '-t', 'auto', u'/dev/xvdb', '/run/cloud-init/tmp/tmplcFFJh'] with allowed return codes [0] (shell=False, capture=True)
2020-07-15 22:12:06,801 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//user-data (quiet=False)
2020-07-15 22:12:06,802 - util.py[DEBUG]: Read 232 bytes from /run/cloud-init/tmp/tmplcFFJh//user-data
2020-07-15 22:12:06,802 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//meta-data (quiet=False)
2020-07-15 22:12:06,803 - util.py[DEBUG]: Read 50 bytes from /run/cloud-init/tmp/tmplcFFJh//meta-data
2020-07-15 22:12:06,803 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//vendor-data (quiet=False)
2020-07-15 22:12:06,803 - util.py[DEBUG]: Reading from /run/cloud-init/tmp/tmplcFFJh//network-config (quiet=False)
2020-07-15 22:12:06,805 - util.py[DEBUG]: Read 104 bytes from /run/cloud-init/tmp/tmplcFFJh//network-config
2020-07-15 22:12:06,805 - util.py[DEBUG]: Running command ['umount', '/run/cloud-init/tmp/tmplcFFJh'] with allowed return codes [0] (shell=False, capture=True)
2020-07-15 22:12:06,825 - util.py[DEBUG]: Attempting to load yaml from string of length 50 with allowed root types (<type 'dict'>,)
2020-07-15 22:12:06,826 - util.py[DEBUG]: Attempting to load yaml from string of length 104 with allowed root types (<type 'dict'>,)
2020-07-15 22:12:06,827 - util.py[DEBUG]: loaded blob returned None, returning default.
2020-07-15 22:12:06,827 - handlers.py[DEBUG]: finish: init-local/search-NoCloud: FAIL: no local data found from DataSourceNoCloud
2020-07-15 22:12:06,827 - util.py[WARNING]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'> failed
2020-07-15 22:12:06,828 - util.py[DEBUG]: Getting data from <class 'cloudinit.sources.DataSourceNoCloud.DataSourceNoCloud'> failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 743, in find_source
    if s.update_metadata([EventType.BOOT_NEW_INSTANCE]):
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 632, in update_metadata
    result = self.get_data()
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/__init__.py", line 256, in get_data
    return_value = self._get_data()
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceNoCloud.py", line 126, in _get_data
    mydata = _merge_new_seed(mydata, seeded)
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceNoCloud.py", line 353, in _merge_new_seed
    util.load_yaml(seeded.get('network-config')))
  File "/usr/lib/python2.7/site-packages/cloudinit/sources/DataSourceNoCloud.py", line 324, in _maybe_remove_top_network
    network_val = cfg.get('network', nullval)
AttributeError: 'NoneType' object has no attribute 'get'
2020-07-15 22:12:06,842 - main.py[DEBUG]: No local datasource found

The full cloud-init.log file is here.

What is the explanation for the ssh key config drive to work and the xen orchestra custom config not?