CoreOS - Docker on XCP-ng working (ish) - Hopefully this helps someone

[jmccoy555]

So after parking CoreOS for nearly a year, and after noticing some edits and commits had been made I decided to give it another go; this time with success…. Yipppeee

• XCP-ng 8.0.0 with latest patches
• CoreOS version is Container Linux 2303.3.0
• XO from sources - xo-server 5.54.0

There are a few buts and pointers though!!

Deploying the VM did not work from XO (from sources, just a homelabber here); the config disc was not created.

So reverting to XCP-ng Centre…….

Following https://github.com/xcp-ng/xcp/wiki/Docker-in-XCP-ng for general guidance – BUT I was unable to log in with my SSH key.

This is fixed by enclosing the key in “ “ thus, my cloud-config was

#cloud-config

hostname: %VMNAMETOHOSTNAME%
ssh_authorized_keys:
  - “ssh-rsa AAAAB3NzaC……………………..syQ4cJYnG70BqQJZVqo5fGcIQP9j5UY1 user@host"
  # - ssh-rsa <Your public key>
  # The following entry will automatically be replaced with a public key
  # generated by container management plugin. The key-entry must exist,
  # in order to enable container management for this VM.
  - ssh-rsa %CONTAINERRSAPUB%
coreos:
  units:
    - name: etcd-member.service
      command: start

    # Hypervisor Linux Guest Agent
    - name: xe-linux-distribution.service
      command: start
      content: |
        [Unit]
        Description=Hypervisor Linux Guest Agent
        After=docker.service

        [Service]
        ExecStartPre=/media/configdrive/agent/xe-linux-distribution /var/cache/xe-linux-distribution
        ExecStart=/media/configdrive/agent/xe-daemon
  etcd:
    name: %VMNAMETOHOSTNAME%
    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
    # specify the initial cluster size using ?size=X
    # discovery: "https://discovery.etcd.io/<token>"
write_files:
  # Enable ARP notifications for smooth network recovery after migrations
  - path: /etc/sysctl.d/10-enable-arp-notify.conf
    permissions: 0644
    owner: root
    content: |
      net.ipv4.conf.all.arp_notify = 1


# Template loaded from /usr/lib/python2.7/site-packages/xscontainer/data/cloud-config.template

Following the rest of the guide everything appeared to go well.

Launching a container and then the next issue, it doesnt show in XO or XCP-ng Centre. Following this post https://xcp-ng.org/forum/topic/1679/how-to-fix-coreos-docker-container-tree-view-in-xcp-ng-center/2 and vola, all working.

Now I have a working mater a fast clone avoids any of these issues for additioanl hosts.

Creating a container without –restart always does not allow me to start / restart / stop it from XO, not the end of the world, looks like this has been found https://xcp-ng.org/forum/topic/2148/can-t-start-docker-container-from-xoa

Happy to try and help resolve any of these 'issues', test and provide logs etc. etc.

[MaisUmGajo]

@jmccoy555 Thanks, I was going mad.

[pnunn]

I tried the commands in "can-t-start-docker-container-from-xoa" but it blew up unfortunately.

xscontainer-prepare-vm -v 0390f015-9302-ccbd-e4af-1bbbba283cfd -u core
Would you like to push a pool-specific public SSH key into the ~/.ssh/authorized_keys file of the specified VM and therefore authorize hosts in the pool to interact with the containers inside the VM?
Answer y/n:
y
Traceback (most recent call last):
  File "/usr/bin/xscontainer-prepare-vm", line 272, in <module>
    sys.exit(main())
  File "/usr/bin/xscontainer-prepare-vm", line 246, in main
    resultcode = _handle_ssh(session, options)
  File "/usr/bin/xscontainer-prepare-vm", line 139, in _handle_ssh
    resultcode = push_ssh_key(session, options.vmuuid, options.username)
  File "/usr/bin/xscontainer-prepare-vm", line 80, in push_ssh_key
    print(docker.ERROR_CAUSE_NETWORK)
AttributeError: 'module' object has no attribute 'ERROR_CAUSE_NETWORK'

Maybe related, when I tried to look at the current sshhostkey it said the parameter didn't exist.

Not too concerned, portainer seems to be the go instead. But thought I'd report my experiences.

[pnunn]

@pnunn Got it....

guest tools hadn't installed correctly.

Manually installed them.. ran the xscontainer-prepare-vm again and now have a container tab on the core OS machine showing the containers running.

P.