XCP-ng 8.3 betas and RCs feedback 🚀

Anonabhar

@pierre-c I dont use iSCSI in my lab anymore, but yesterday when I did a "yum update" of 8.3 Beta, I got a script error when yum was running through all the updates. The script that failed was part of the sm update.

To be honest, I didnt bother doing anything with the error as everything seemed to work OK for me after, so I just ignored it. But.. Maybe I should have said something.. Sorry guys..

stormi

@Anonabhar We discussed the script error above. It's benign and will be fixed.

brezlord

@brezlord said in XCP-ng 8.3 betas and RCs feedback :

I did an update of a 8.2 host to 8.3 via ISO install. Everything is working as it should but I get the below error on the host advanced tab with PCI passthrough. I had a Nvidia GPU passed through to a RHEL 9 VM on the 8.2 host. This was done via the command line.

I will load 8.3 on a new host with the same hardware config later in the week to confirm that it must be something to do with the 8.2 --> 8.3 upgrade.

Let me know if you need more info.

@olivierlambert I can confirm that a host with the exact same hardware config and a fresh install of 8.3-RC does not have this issue. Only the host upgraded from 8.2 LTS.

olivierlambert

That's interesting. Thanks for the feedback!

brezlord

@olivierlambert Do you need any further information before I re-image with the host 8.3?

ecoutinho

While installing 8.3 RC1 it says BIOS boot mode is deprecated and I should use UEFI. But if I boot in UEFI, when I choose to upgrade the existing installation, an error occurs: "Installer mode (UEFI) is mismatched with target host mode (legacy)".

To be clear, it is not possible to migrate from BIOS mode to UEFI during an upgrade?

In that case, the only option would be to remove the host from the pool, install XCP-ng (not upgrade) and then add it to the pool. Am I correct? Thanks.

stormi

@ecoutinho at the moment, reinstalling is indeed the only solution to switch from BIOS to UEFI.

Note that the deprecation announcement comes with 8.3, but BIOS boot mode will still work and is still tested for this release. So you have some time to plan the switch.

ecoutinho

@stormi I've upgraded the master in BIOS mode, and proceeded to reinstall one of the other hosts in UEFI mode, after having removed it from the pool. It went fine, until I tried to add the host back into the pool. This was not possible because the reinstalled host has Certificate Verification enabled, while the pool doesn't. Even if I host-emergency-disable-tls-verification, it's not possible to add the server back into the pool.

It seems I'll have to upgrade the other hosts in BIOS mode, enable certificate verification on the pool and then add this host. I guess I'll reinstall the other hosts in UEFI mode on a future upgrade.

stormi

@ecoutinho You can enable TLS verification on the pool then join the new host. Or disable it on the new host but that's a downgrade of this new security feature meant to protect against MITM attacks.

ecoutinho

@stormi Thanks for your suggestions. I've tried to enable it on the pool:

# xe pool-enable-tls-verification
This operation is not supported during an upgrade.

I have to finish the upgrade of the other hosts before enabling it on the pool.

As for disabling it on the new host, I didn't find any way to do it permanently. I just found the host-emergency-disable-tls-verification option, which does not disable it completely, and doesn't allow to add it to a pool without TLS verification. Would you clarify how to disable it on the new host?

I will enable it on the pool when the upgrade is finished.

stormi

@ecoutinho I don't see a way to disable TLS verification, but anyway I wouldn't join a host to a pool which is currently being upgraded. I even suspect XAPI would refuse.

ecoutinho

@stormi OK, thanks, I'll finish upgrading the other hosts.

stormi

@brezlord If you haven't reinstalled it yet, yes, we could use more information about the host, the PCI passthrough setup on it, how the error is triggered exactly, and various logs. And/or a support tunnel to have a look by ourselves.

BenjiReis

@brezlord Can you sahre the value you put in the xen-cmdline on 8.2?

I think the XAPI awaits this format: xen-pciback.hide=(pci-id1)(pci-id2)... and will fail the value doesn't match this format.

brezlord

@BenjiReis Sorry I have destroyed the host and loaded a fresh install of 8.3. If you'd like I can install 8.2 on a host and pass through a pci device via the cmd then upgrade to 8.3 and see if the error is reproducible.

BenjiReis

@brezlord If it's not to much a bother that would be great yeah.
Comparing the xen-cmdline when doing the passthrough manually on 8.2 VS how it looks on 8.3 and when done via the XAPI.

stormi

I'm pushed a new set of updates, hopefully the last one before the release of XCP-ng 8.3.0 RC2, which itself should be followed shortly by the release of XCP-ng 8.3.0 itself.

Main packages

intel-microcode-20240717-1.xcpng8.3: updated microcode for Intel vulnerabilities
sm-3.2.3-1.4.xcpng8.3: fix the cause of a warning displayed during update, and restore changes that we had removed because they were suspected to cause issues in some cases with iSCSI, but revealed themselves necessary to support another kind of setup.
vim-7.4.629-8.el7_9 (which provides vim-minimal, installed by default): bugfixes and security fixes
xapi-24.19.2-1.3.xcpng8.3: Fixes an issue where new fields in XAPI DB for certificate fingerprints were not populated, which under some circumstances caused joining new hosts to a pool fail.
xcp-ng-release-8.3.0-28:
- Update repository files for CentOS and EPEL.
- Point at repo.vates.tech for CentOS since mirrorlist.centos.org was cut
- Add "(EOL)" to repo descriptions for EOL repos
- Drop unused repos

Optional package

kernel-alt-4.19.316+1-2.xcpng8.3: Enable CONFIG_X86_AMD_PLATFORM_DEVICE in kernel config
ldns-1.7.0-21.xcpng8.3 + libreswan-4.12-2.3.1.xcpng8.3: security updates

Ajmind 0

On my 8.3 test pool I am unable to create SR ISO libary (SMB/cifs).

On my production pools with XCP-NG 8.2.x it does work as expected.

olivierlambert

Can you provide a bit more details? Errors and such.

Ajmind 0

@olivierlambert
yes I can

on dom0:

mount -t cifs --verbose -o username=admin,password=******** //192.168.1.202/iso /mnt/test
mount.cifs kernel mount options: ip=192.168.1.202,unc=\\192.168.1.202\iso,user=admin,pass=********
mount error(112): Host is down

XCP-NG Center:

Creating ISO SR 'SMB ISO library' on 'IT1HALIZARD-TEST1'
Unable to mount the directory specified in device configuration request	it1xcp-ng-test-slave1	Sep 5, 2024 12:59 PM	Dismiss

XO from Sources:

sr.createIso
{
  "host": "c1f34b07-c4dc-4584-8bc0-a01bcec81c5b",
  "nameLabel": "test",
  "nameDescription": "test",
  "path": "\\\\192.168.1.202\\public\\iso",
  "type": "smb",
  "user": "admin",
  "password": "* obfuscated *"
}
{
  "code": "SR_BACKEND_FAILURE_222",
  "params": [
    "",
    "Could not mount the directory specified in Device Configuration [opterr=mount error(112): Host is down
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)]",
    ""
  ],
  "call": {
    "method": "SR.create",
    "params": [
      "OpaqueRef:cee521f7-dc00-5d91-1499-6143d2fd0040",
      {
        "type": "cifs",
        "username": "admin",
        "cifspassword": "* obfuscated *",
        "location": "//192.168.1.202/public/iso"
      },
      0,
      "test",
      "test",
      "iso",
      "iso",
      true,
      {}
    ]
  },
  "message": "SR_BACKEND_FAILURE_222(, Could not mount the directory specified in Device Configuration [opterr=mount error(112): Host is down
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)], )",
  "name": "XapiError",
  "stack": "XapiError: SR_BACKEND_FAILURE_222(, Could not mount the directory specified in Device Configuration [opterr=mount error(112): Host is down
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)], )
    at Function.wrap (file:///srv/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
    at file:///srv/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21
    at runNextTicks (node:internal/process/task_queues:60:5)
    at processImmediate (node:internal/timers:447:9)
    at process.callbackTrampoline (node:internal/async_hooks:128:17)"
}

The SR is mounted on our production pools.