TLS failure during VxLAN creation

dgmcdona

When attempting to create a VxLAN, I am receiving the following error:

Client network socket disconnected before secure TLS connection was established

Examination of packet capture reveals that the hosts are returning a fatal(48) TLS error, and disconnecting. SDN-Controller plugin is set to create its own certificates, located in /var/lib/xo-server/data/sdn-controller/ directory.

Our configuration is as follows:
5 hosts running xcp-ng 8.1
Latest Xen Orchestra release (5.50) compiled from sources on Debian 10. XO is installed as root, and is running HTTPS on port 443.
Network switch allowing jumbo frames, so an MTU of 1550 can be specified during VxLAN creation.

I have tried copying the ca-cert.pem file to each of the hosts, and adding it as a trusted CA, but this has not solved the problem.

olivierlambert

I thought we solved the issue. Ping @BenjiReis

BenjiReis

Hello,

Is the override-certs settings set to ON in the sdn-controller configuration?
If so, please share XO logs of the failed network creation.

Thanks

dgmcdona

cert-dir is left blank, and override-certs is set to ON.

sdnController.createPrivateNetwork
{
  "poolIds": [
    "847ffc73-e664-9dd6-87bf-c7561ea0c44b"
  ],
  "pifIds": [
    "8bb22ef0-208e-7ab4-737f-b44855da54b0"
  ],
  "name": "VxLAN",
  "description": "Cross-server private network",
  "encapsulation": "vxlan",
  "encrypted": false,
  "mtu": 1550
}
{
  "code": "ECONNRESET",
  "host": "137.30.124.81",
  "port": 6640,
  "message": "Client network socket disconnected before secure TLS connection was established",
  "name": "Error",
  "stack": "Error: Client network socket disconnected before secure TLS connection was established
    at connResetException (internal/errors.js:610:14)
    at TLSSocket.onConnectEnd (_tls_wrap.js:1546:19)
    at Object.onceWrapper (events.js:421:28)
    at TLSSocket.apply (events.js:327:22)
    at TLSSocket.patchedEmit (/opt/xo/xo-builds/xen-orchestra-202027081251/@xen-orchestra/log/src/configure.js:95:17)
    at TLSSocket.EventEmitter.emit (domain.js:483:12)
    at endReadableNT (_stream_readable.js:1220:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)"
}

BenjiReis

Okay, is port 6640 open on our hosts?
can you post the result of iptables --list here?

dgmcdona

Chain INPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_input_chain  tcp  --  anywhere             anywhere             tcp dpt:nbd
xapi-INPUT  all  --  anywhere             anywhere
ACCEPT     gre  --  anywhere             anywhere
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_output_chain  tcp  --  anywhere             anywhere             tcp spt:nbd

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere             icmp any
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:ha-cluster
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:21064
ACCEPT     udp  --  anywhere             anywhere             multiport dports hpoms-dps-lstn,netsupport
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain xapi-INPUT (1 references)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:4789
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:6640
RETURN     all  --  anywhere             anywhere

Chain xapi_nbd_input_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain xapi_nbd_output_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

BenjiReis

Okay so the port is opened. That's weird, can you reload the sdn-controller plugin? It will retry to install the TLS certifiactes and perhaps something goes wrong their, can you get the XO logs when the plugin is reloaded?

dgmcdona

Reloading the SDN controller plugin fixed the problem. Thanks!

BenjiReis

Glad to hear it!

There were probably an issue when updating the SDN controller's TLS certificate.
The reload had solved it.

Regards!