@Tristis-Oris said in What metadata restore really do?:

My pool died so i decied to restore it from backup.

• fresh xen install, retore metadata, everything looks fine, but i can't join any new hosts into this pool.
  https://paste.vates.tech/?ca81f43fd2d10456#Ay6yKRNTS8LDDSHTkin7eLppP33eAMS5grLrevuqf5rL

It looks like the metadata restored has a certificate with name 'sdn-controller-ca', but this certificate does not exist in the filesystem (/etc/stunnel/certs/sdn-controller-ca.pem). I believe this is installed by Xen Orchestra whenever the SDN controller is turned on.
To remove it from the database run xe pool-uninstall-ca-certificate name=sdn-controller-ca --force

The force flag should allow the certificate to be removed from the database even if the file is missing

@rmaclachlan This looks awfully similar to https://github.com/xapi-project/xen-api/pull/5451

freddy77 opened this pull request in xapi-project/xen-api

closed CP-47754: Do not report errors attempting to read PCI vendor:product #5451

@stormi said in XCP-ng 8.3 beta :

@psafont Will a 8.2 to 8.3 upgrade (through the installation ISO) leave TLS verification disabled, or will it enable it by default?

It's not enabled by default, enabling it by default is not possible with the current update procedure where the pool coordinator is updated before the other pool member because these do not expose the new certificates to xmlrpc clients before upgrading, breaking communications.

In other words: must we expect any user who upgrades from 8.2 or lower and then later wants to add a new host to the pool to see this error (and likely ask for help, even if we document it properly - and we would of course)?

Yes

@gsrfan01 The error happens because the joining host has TLS certificate checking enabled for pool connections while the joined host don't.

This mismatch happens because on fresh installs TLS certificate checking is enabled, while for updates from previous versions is not.

To enable TLS certificate checking in a pool simply running xe pool-enable-tls-verification.

The emergency command is not needed in this case, it's useful to re-enable certificate checking in a single host after is has been disabled using the emergency disable

@olivierlambert Snapshot is in essence a VDI clone, I don't see any checks being done before the filtering for ignored VDIs is done. And that is done pretty early on, not sure why there are operations affecting virtual block devices from ignored VDIs: https://github.com/xapi-project/xen-api/blob/master/ocaml/xapi/xapi_vm_clone.ml#L416

@cocoon
The best thing we can do here is inspect the actual certificate:
Please run openssl x509 -text -noout -in /etc/xensource/xapi-ssl.pem

xenserver has generated host certificates with 2048-bit RSA keys for years, these should be able to be loaded by stunnel (through openssl) just fine.

If the key is smaller that this then the fix is easy: generate a new certificate for that host: xe host-refresh-server-certificate host uuid=<>
Be mindful that clients that trusted the previous certificate will need to trust the new one in order for the TLS connections to be established

@bogikornel This is a known issue. The fix has been merged upstream and the plan is to integrate it into xcp-ng in the coming months.

@nikade We've already fixed some issues to start using ocaml 5 regarding the C interfaces. Handling threading in ocaml 5 is still an open problem that the ecosystem has not yet solved (there are many libraries competing now). We still need to create a credible strategy to port xapi to the new model, and don't have any timelines yet

@john-c This involves waiting on a stable OCaml 5 version being release, then porting the whole codebase to use the new multithreading APIs, this is a very complex matter, and it will take a long time until it's realised

@Tristis-Oris said in What metadata restore really do?:

My pool died so i decied to restore it from backup.

• fresh xen install, retore metadata, everything looks fine, but i can't join any new hosts into this pool.
  https://paste.vates.tech/?ca81f43fd2d10456#Ay6yKRNTS8LDDSHTkin7eLppP33eAMS5grLrevuqf5rL

It looks like the metadata restored has a certificate with name 'sdn-controller-ca', but this certificate does not exist in the filesystem (/etc/stunnel/certs/sdn-controller-ca.pem). I believe this is installed by Xen Orchestra whenever the SDN controller is turned on.
To remove it from the database run xe pool-uninstall-ca-certificate name=sdn-controller-ca --force

The force flag should allow the certificate to be removed from the database even if the file is missing

@bogikornel This is a known issue. The fix has been merged upstream and the plan is to integrate it into xcp-ng in the coming months.

@MJGZ51 said in XCP-ng 8.3 PCI Passthrough issue:

Does anyone have any ideas what the error shown above is from?

It's generated by OCaml code, but it's difficult to tell where from without a backtrace that should be found in /var/log/xensource.log

@olivierlambert I don't think the first was ever meant to support that. Without knowing how much effort it will be, I'm leaning towards the second option, to not reset the CBT.

@olivierlambert xe vm-migrate uses migrate_send when storage or network needs to be changed, otherwise vm.pool_migrate is used. Selecting a new network is done through a the vif parameter. This parameter is a map in the form of vif:<VIF_UUID>=<NEW_NETWORK_UUID> vif:<VIF_UUID2>=<NEW_NETWORK_UUID2> (and so on).

So I'm not so sure that a netwrok migration can happen when using pool_migrate.

@nikade We've already fixed some issues to start using ocaml 5 regarding the C interfaces. Handling threading in ocaml 5 is still an open problem that the ecosystem has not yet solved (there are many libraries competing now). We still need to create a credible strategy to port xapi to the new model, and don't have any timelines yet

@rmaclachlan This looks awfully similar to https://github.com/xapi-project/xen-api/pull/5451

freddy77 opened this pull request in xapi-project/xen-api

closed CP-47754: Do not report errors attempting to read PCI vendor:product #5451

@john-c This involves waiting on a stable OCaml 5 version being release, then porting the whole codebase to use the new multithreading APIs, this is a very complex matter, and it will take a long time until it's realised

@stormi said in XCP-ng 8.3 beta :

@psafont Will a 8.2 to 8.3 upgrade (through the installation ISO) leave TLS verification disabled, or will it enable it by default?

It's not enabled by default, enabling it by default is not possible with the current update procedure where the pool coordinator is updated before the other pool member because these do not expose the new certificates to xmlrpc clients before upgrading, breaking communications.

In other words: must we expect any user who upgrades from 8.2 or lower and then later wants to add a new host to the pool to see this error (and likely ask for help, even if we document it properly - and we would of course)?

Yes

@gsrfan01 The error happens because the joining host has TLS certificate checking enabled for pool connections while the joined host don't.

This mismatch happens because on fresh installs TLS certificate checking is enabled, while for updates from previous versions is not.

To enable TLS certificate checking in a pool simply running xe pool-enable-tls-verification.

The emergency command is not needed in this case, it's useful to re-enable certificate checking in a single host after is has been disabled using the emergency disable