@pdonias One of my VMs isn't reporting an IP address. Do you think that could do it?

Same as the OP.

netbox.synchronize
{
  "pools": [
    "582481e9-4e61-a060-6c09-530e35606504"
  ]
}
{
  "message": "ipaddr: the address has neither IPv6 nor IPv4 format",
  "name": "Error",
  "stack": "Error: ipaddr: the address has neither IPv6 nor IPv4 format
    at Object.parse (/opt/xo/xo-builds/xen-orchestra-202112260451/packages/xo-server-netbox/node_modules/ipaddr.js/lib/ipaddr.js:929:19)
    at Netbox.#synchronize (/opt/xo/xo-builds/xen-orchestra-202112260451/packages/xo-server-netbox/src/index.js:560:35)
    at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202112260451/packages/xo-server/src/xo-mixins/api.mjs:307:20)"
}

Its the 5.66.1 release commit-so its damn close.

I think so? Unless my script isn't updating the plugins, which is totally possible.

edit: quick look at github says my plugins are fine

@notscottsmith @pdonias

any updates on this? Currently running into this error myself.

@ronan-a oop-good to know. Now I guess I need to figure out why the new image i created is exiting instead of, well, working.

Unless there's something in this command that I shouldn't be invoking.

podman create --health-cmd="wget --no-verbose --tries=1 --spider http://127.0.0.1:8080/ || exit 1" --volume=/root/mjolnir:/data:Z matrixdotorg/mjolnir

(I start it separately, later)

Oh now that's interesting. Turns out the containers (both archlinux and the one i just created) are exiting w/ error 143. They're getting sigterm'ed from somewhere.

For what its worth, the podman logs archlinux command from above is w/o debug. I didn't quite realize it vanishing immediately was intended behavior though, tells you how versed I am in containers.

I'll try setting up the matrixdotorg/mjolnir thing again now that I have a command I know is working on runc.

@ronan-a Confirmed all three points. I did set GUEST_TOOLS in runx.conf-I do wonder if that could be the problem here.

For what its worth log-level debug shouldn't ever make it over to runx, so those earlier pastes should effectively be podman start archlinux

edit: a quick test says that GUEST_TOOLS isnt it.

edit: more logs for ya:

[04:01 lenovo150 ~]# podman logs archlinux
connect to container console with 'xl console abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4'
mount: mount(2) failed: Not a directory
mount: mount(2) failed: Not a directory
mkdir: cannot create directory '/var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/merged/etc/hosts': File exists
mount: mount(2) failed: Not a directory
rm: cannot remove '/var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/merged//etc/hosts': Device or resource busy
cp: '/var/run/containers/storage/overlay-containers/abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4/userdata/hosts' and '/var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/merged//etc/hosts' are the same file

Worth noting that I have a network adapter on the template-if i run it without a network adapter it still fails, but slightly differently.

DEBU[0000] Reading configuration file "/etc/containers/libpod.conf"
DEBU[0000] Merged system config "/etc/containers/libpod.conf": &{{false false false false false true} 0 {   [] [] []}  docker://  runx map[crun:[/usr/bin/crun /usr/local/bin/crun] runc:[/usr/bin/runc /usr/sbin/runc /usr/local/bin/runc /usr/local/sbin/runc /sbin/runc /bin/runc /usr/lib/cri-o-runc/sbin/runc /run/current-system/sw/bin/runc] runx:[/usr/bin/runx]] [runx] [runx] [] [/usr/libexec/podman/conmon /usr/local/libexec/podman/conmon /usr/local/lib/podman/conmon /usr/bin/conmon /usr/sbin/conmon /usr/local/bin/conmon /usr/local/sbin/conmon /run/current-system/sw/bin/conmon] [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] systemd   /var/run/libpod -1 false /etc/cni/net.d/ [/usr/libexec/cni /usr/lib/cni /usr/local/lib/cni /opt/cni/bin] runx []   k8s.gcr.io/pause:3.1 /pause false false  2048 shm    false}
DEBU[0000] Reading configuration file "/usr/share/containers/libpod.conf"
DEBU[0000] Merged system config "/usr/share/containers/libpod.conf": &{{false false false false false true} 0 {   [] [] []}  docker://  runx map[crun:[/usr/bin/crun /usr/local/bin/crun] runc:[/usr/bin/runc /usr/sbin/runc /usr/local/bin/runc /usr/local/sbin/runc /sbin/runc /bin/runc /usr/lib/cri-o-runc/sbin/runc /run/current-system/sw/bin/runc] runx:[/usr/bin/runx]] [runx] [runx] [] [/usr/libexec/podman/conmon /usr/local/libexec/podman/conmon /usr/local/lib/podman/conmon /usr/bin/conmon /usr/sbin/conmon /usr/local/bin/conmon /usr/local/sbin/conmon /run/current-system/sw/bin/conmon] [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] systemd   /var/run/libpod -1 false /etc/cni/net.d/ [/usr/libexec/cni /usr/lib/cni /usr/local/lib/cni /opt/cni/bin] runx []   k8s.gcr.io/pause:3.1 /pause false false  2048 shm    false}
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /var/lib/containers/storage
DEBU[0000] Using run root /var/run/containers/storage
DEBU[0000] Using static dir /var/lib/containers/storage/libpod
DEBU[0000] Using tmp dir /var/run/libpod
DEBU[0000] Using volume path /var/lib/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] cached value indicated that overlay is supported
DEBU[0000] cached value indicated that metacopy is not being used
DEBU[0000] cached value indicated that native-diff is usable
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false
DEBU[0000] Initializing event backend journald
WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument
DEBU[0000] using runtime "/usr/bin/runx"
DEBU[0000] using runtime "/usr/bin/runc"
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist
INFO[0000] Found CNI network runx (type=loopback) at /etc/cni/net.d/99-podman-runx.conflist
DEBU[0000] Made network namespace at /var/run/netns/cni-ad3f7be8-7341-a2d3-fd7d-9e417dd8c888 for container abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4
INFO[0000] Got pod network &{Name:archlinux Namespace:archlinux ID:abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 NetNS:/var/run/netns/cni-ad3f7be8-7341-a2d3-fd7d-9e417dd8c888 Networks:[] RuntimeConfig:map[runx:{IP: PortMappings:[] Bandwidth:<nil> IpRanges:[]}]}
INFO[0000] About to add CNI network cni-loopback (type=loopback)
DEBU[0000] overlay: mount_data=lowerdir=/var/lib/containers/storage/overlay/l/57IKSKIB7PWKFFISHSGCHN7OJL:/var/lib/containers/storage/overlay/l/UGY43VPEWOHWBCZOQR4WMP5A6F,upperdir=/var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/diff,workdir=/var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/work
DEBU[0000] mounted container "abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4" at "/var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/merged"
DEBU[0000] Created root filesystem for container abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 at /var/lib/containers/storage/overlay/742626ef59426855d765f2cee7b24cac06ecacc60c5ae37668d1f95ff649cd22/merged
INFO[0000] Got pod network &{Name:archlinux Namespace:archlinux ID:abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 NetNS:/var/run/netns/cni-ad3f7be8-7341-a2d3-fd7d-9e417dd8c888 Networks:[] RuntimeConfig:map[runx:{IP: PortMappings:[] Bandwidth:<nil> IpRanges:[]}]}
INFO[0000] About to add CNI network runx (type=loopback)
DEBU[0000] [0] CNI result: Interfaces:[{Name:lo Mac:00:00:00:00:00:00 Sandbox:/var/run/netns/cni-ad3f7be8-7341-a2d3-fd7d-9e417dd8c888}], IP:[{Version:4 Interface:0xc0001d8bf0 Address:{IP:127.0.0.1 Mask:ff000000} Gateway:<nil>} {Version:6 Interface:0xc0001d8c28 Address:{IP:::1 Mask:ffffffffffffffffffffffffffffffff} Gateway:<nil>}], DNS:{Nameservers:[] Domain: Search:[] Options:[]}
DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode secret
DEBU[0000] Setting CGroups for container abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 to machine.slice:libpod:abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4
DEBU[0000] reading hooks from /usr/share/containers/oci/hooks.d
DEBU[0000] added hook /usr/share/containers/oci/hooks.d/oci-register-machine.json
DEBU[0000] added hook /usr/share/containers/oci/hooks.d/oci-systemd-hook.json
DEBU[0000] added hook /usr/share/containers/oci/hooks.d/oci-umount.json
DEBU[0000] hook oci-register-machine.json did not match
DEBU[0000] hook oci-systemd-hook.json did not match
DEBU[0000] hook oci-umount.json did not match
DEBU[0000] reading hooks from /etc/containers/oci/hooks.d
DEBU[0000] Created OCI spec for container abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 at /var/lib/containers/storage/overlay-containers/abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4/userdata/config.json
DEBU[0000] /usr/bin/conmon messages will be logged to syslog
DEBU[0000] running conmon: /usr/bin/conmon               args="[--api-version 1 -s -c abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 -u abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 -r /usr/bin/runx -b /var/lib/containers/storage/overlay-containers/abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4/userdata -p /var/run/containers/storage/overlay-containers/abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4/userdata/pidfile -l k8s-file:/var/lib/containers/storage/overlay-containers/abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4/userdata/ctr.log --exit-dir /var/run/libpod/exits --socket-dir-path /var/run/libpod/socket --log-level debug --syslog --conmon-pidfile /var/run/containers/storage/overlay-containers/abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /var/run/containers/storage --exit-command-arg --log-level --exit-command-arg error --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /var/run/libpod --exit-command-arg --runtime --exit-command-arg runx --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4]"
INFO[0000] Running conmon under slice machine.slice and unitName libpod-conmon-abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4.scope
DEBU[0000] Cleaning up container abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4
DEBU[0000] Tearing down network namespace at /var/run/netns/cni-ad3f7be8-7341-a2d3-fd7d-9e417dd8c888 for container abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4
INFO[0000] Got pod network &{Name:archlinux Namespace:archlinux ID:abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4 NetNS:/var/run/netns/cni-ad3f7be8-7341-a2d3-fd7d-9e417dd8c888 Networks:[] RuntimeConfig:map[runx:{IP: PortMappings:[] Bandwidth:<nil> IpRanges:[]}]}
INFO[0000] About to del CNI network runx (type=loopback)
DEBU[0000] unmounted container "abb22f4d68083424252ac7116427dfd9c3644291e858039750f135cae499f8b4"
ERRO[0000] unable to start container "archlinux": container create failed (no logs from conmon): EOF