Hi @BenjiReis, the certificate procedure I posted yesterday isn't quite right and although the plugin will enable, TLS will not negotiate. My openssl.conf defaults to adding x509v3 attributes that breaks the TLS negotiation process.

Here's the correct procedure on Rocky Linux 9. Should work for most other recent Linux flavors.

Create a temporary directory to work in, and copy your OS's openssl.cnf file into it. Edit the [ v3_ca ] section so it only has the following 2 entries:

subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer

Now create the CA (run all commands in your tmp directory). I am also now using 4096 bit strength:

openssl genrsa 4096 > ca-key.pem
openssl req -new -x509 -config ./openssl.cnf -nodes -days 365000 -key ca-key.pem -out ca-cert.pem

** The server certificate step in the previous post is not necessary so just skip it.

Now create a file called client_attr.cnf and put the following entries in it:

subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer

Now Create the client certificate:

openssl req -newkey rsa:4096 -nodes -keyout client-key.pem -out client-req.pem
openssl x509 -req -days 365000 -set_serial 01 -in client-req.pem -out client-cert.pem -CA ca-cert.pem -CAkey ca-key.pem -extfile ./client_attr.cnf

The ca-cert.pem and client_cert.pem should now conform to the correct x509 format that will work with XOA and hence cross-server private networking.

Sorry for the incorrect procedure, it was a late night

Cheers.

@BenjiReis Here's the original blog post. Apologies for any formatting issues:

There seems to be a bug in the SDN Controller plugin in Xen Orchestra when building and installing the community version. The bug seems to go back a few commit versions. When enabling the plugin and allowing the plugin to generate its own certificates, you'll get an error something like this:

plugin.load
{
"id": "sdn-controller"
}
{
"code": 10,
"data": {},
"message": "plugin not configured",
"name": "XoError",
"stack": "XoError: plugin not configured
at invalidParameters (/opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-common/api-errors.js:26:11)
at default.loadPlugin (file:///opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-server/src/xo-mixins/plugins.mjs:210:13)
at Xo.call (file:///opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-server/src/api/plugin.mjs:61:14)
at Api.#callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-server/src/xo-mixins/api.mjs:394:33)"
}

I had an older version of XOA that I built on Rocky Linux 8.6 that did have a working SDN controller plugin and it was enabled. I decided to do some sleuthing and found that the cause is a missing client-cert.pem file in the plugin directory, /var/lib/xo-server/data/sdn-controller. The plugin should have three files in that directory: client-cert.pem, client-key.pem and ca-cert.pem. As for the root cause of this, I didn't dig any further. I was installing XOA on Rocky OS 9 and that may be a factor.

Either way, there is a fix and that is to simply regenerate those three required files, and flip on the 'override-certs' toggle switch. Here's the procedure. Note you could use another certificate authority, but for completeness we'll just create one. In a temp directory of your choosing execute the following:

Create the CA certificate/key:

openssl genrsa 2048 > ca-key.pem
openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca-cert.pem

Create the server certificate/key:

openssl req -newkey rsa:2048 -nodes -days 365000 -keyout server-key.pem -out server-req.pem
openssl x509 -req -days 365000 -set_serial 01 -in server-req.pem -out server-cert.pem -CA ca-cert.pem -CAkey ca-key.pem

Create the client certificate/key:

openssl req -newkey rsa:2048 -nodes -days 365000 -keyout server-key.pem -out server-req.pem
openssl x509 -req -days 365000 -set_serial 01 -in server-req.pem -out server-cert.pem -CA ca-cert.pem -CAkey ca-key.pem

Now, copy the 3 generated files client-cert.pem, ca-cert.pem and client-key.pem over to the directory /var/lib/xo-server/data/sdn-controller. Then in XOA, under Settings | Plugins, click the plus sign for the sdn-controller plugin and set the override-certs toggle to on and save the configuration.

Next, enable the sdn-controller plugin. You should no longer get the error message above and cross server private networking should now work.

Note that I tested this on Rocky Linux 9, Node 18.12.1, XCP-NG 8.2.1 LTS and tried commits 2b1ed (older) and 57235 (master) for the community build.

If you get an error similar to 'Client network socket disconnected before secure TLS connection was established' when actually creating the private network, you likely have a cipher negotiation problem. See https://github.com/vatesfr/xen-orchestra/issues/5074 for possible workarounds for that particular issue.

Cheers!

ismo-conguairta created this issue in vatesfr/xen-orchestra

closed [xo-server-sdn-controller] Error while creating tunnel - ovsdb-client ERROR TLS connection failed #5074

Hi @BenjiReis, the certificate procedure I posted yesterday isn't quite right and although the plugin will enable, TLS will not negotiate. My openssl.conf defaults to adding x509v3 attributes that breaks the TLS negotiation process.

Here's the correct procedure on Rocky Linux 9. Should work for most other recent Linux flavors.

Create a temporary directory to work in, and copy your OS's openssl.cnf file into it. Edit the [ v3_ca ] section so it only has the following 2 entries:

subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid:always,issuer

Now create the CA (run all commands in your tmp directory). I am also now using 4096 bit strength:

openssl genrsa 4096 > ca-key.pem
openssl req -new -x509 -config ./openssl.cnf -nodes -days 365000 -key ca-key.pem -out ca-cert.pem

** The server certificate step in the previous post is not necessary so just skip it.

Now create a file called client_attr.cnf and put the following entries in it:

subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer

Now Create the client certificate:

openssl req -newkey rsa:4096 -nodes -keyout client-key.pem -out client-req.pem
openssl x509 -req -days 365000 -set_serial 01 -in client-req.pem -out client-cert.pem -CA ca-cert.pem -CAkey ca-key.pem -extfile ./client_attr.cnf

The ca-cert.pem and client_cert.pem should now conform to the correct x509 format that will work with XOA and hence cross-server private networking.

Sorry for the incorrect procedure, it was a late night

Cheers.

@BenjiReis Here's the original blog post. Apologies for any formatting issues:

There seems to be a bug in the SDN Controller plugin in Xen Orchestra when building and installing the community version. The bug seems to go back a few commit versions. When enabling the plugin and allowing the plugin to generate its own certificates, you'll get an error something like this:

plugin.load
{
"id": "sdn-controller"
}
{
"code": 10,
"data": {},
"message": "plugin not configured",
"name": "XoError",
"stack": "XoError: plugin not configured
at invalidParameters (/opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-common/api-errors.js:26:11)
at default.loadPlugin (file:///opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-server/src/xo-mixins/plugins.mjs:210:13)
at Xo.call (file:///opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-server/src/api/plugin.mjs:61:14)
at Api.#callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202211061813/packages/xo-server/src/xo-mixins/api.mjs:394:33)"
}

I had an older version of XOA that I built on Rocky Linux 8.6 that did have a working SDN controller plugin and it was enabled. I decided to do some sleuthing and found that the cause is a missing client-cert.pem file in the plugin directory, /var/lib/xo-server/data/sdn-controller. The plugin should have three files in that directory: client-cert.pem, client-key.pem and ca-cert.pem. As for the root cause of this, I didn't dig any further. I was installing XOA on Rocky OS 9 and that may be a factor.

Either way, there is a fix and that is to simply regenerate those three required files, and flip on the 'override-certs' toggle switch. Here's the procedure. Note you could use another certificate authority, but for completeness we'll just create one. In a temp directory of your choosing execute the following:

Create the CA certificate/key:

openssl genrsa 2048 > ca-key.pem
openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca-cert.pem

Create the server certificate/key:

openssl req -newkey rsa:2048 -nodes -days 365000 -keyout server-key.pem -out server-req.pem
openssl x509 -req -days 365000 -set_serial 01 -in server-req.pem -out server-cert.pem -CA ca-cert.pem -CAkey ca-key.pem

Create the client certificate/key:

openssl req -newkey rsa:2048 -nodes -days 365000 -keyout server-key.pem -out server-req.pem
openssl x509 -req -days 365000 -set_serial 01 -in server-req.pem -out server-cert.pem -CA ca-cert.pem -CAkey ca-key.pem

Now, copy the 3 generated files client-cert.pem, ca-cert.pem and client-key.pem over to the directory /var/lib/xo-server/data/sdn-controller. Then in XOA, under Settings | Plugins, click the plus sign for the sdn-controller plugin and set the override-certs toggle to on and save the configuration.

Next, enable the sdn-controller plugin. You should no longer get the error message above and cross server private networking should now work.

Note that I tested this on Rocky Linux 9, Node 18.12.1, XCP-NG 8.2.1 LTS and tried commits 2b1ed (older) and 57235 (master) for the community build.

If you get an error similar to 'Client network socket disconnected before secure TLS connection was established' when actually creating the private network, you likely have a cipher negotiation problem. See https://github.com/vatesfr/xen-orchestra/issues/5074 for possible workarounds for that particular issue.

Cheers!

ismo-conguairta created this issue in vatesfr/xen-orchestra

closed [xo-server-sdn-controller] Error while creating tunnel - ovsdb-client ERROR TLS connection failed #5074

Hi @BenjiReis,

We have quite a restrictive firewall due to hosting for some Canadian customers, so I will post to LinkedIn and to this forum if possible, shortly.

What country are you accessing from?

Thanks.

@morib You are correct, the sdn-controller plugin seems to be broken going back a few commits. I encountered this recently and it's still a problem with the latest master.

TLDR;

The plugin fails to generate the client-cert.pem file located in the directory /var/lib/xo-server/data/sdn-controller.

I posted a blog, describing the error and a solution to get it working here.

Although the original question was about a single server private network, I though this info would be useful for others trying to use the sdn-controller plugin for cross server private networking as well.

Cheers.