Has this been fixed by the update described here?
https://xcp-ng.org/blog/2026/05/21/may-2026-updates-3-for-xcp-ng-8-3-lts/?ref=xen-orchestra.com
Posts
-
RE: Test results for Dell Poweredge R770 with NVMe drives
-
RE: Test results for Dell Poweredge R770 with NVMe drives
@yannsionneau
Thank you.I tested it and I can confirm this solves the issue.
Hope you can release the fix quickly. -
RE: Test results for Dell Poweredge R770 with NVMe drives
Also works with alt driver (broadcom-bnxt-en-alt-1.10.3_231.0.162.0-1.xcpng8.2.x86_64) in 8.2.1:
# yum install broadcom-bnxt-en-alt # reboot # modinfo bnxt_en filename: /lib/modules/4.19.0+1/override/bnxt_en.ko version: 1.10.3-231.0.162.0 description: Broadcom NetXtreme-C/E/S network driver license: GPL srcversion: 533BB7E5866E52F63B9ACCB alias: pci:v000014E4d0000D800sv*sd*bc*sc*i* ... alias: pci:v000014E4d00001604sv*sd*bc*sc*i* depends: devlink,ipv6 retpoline: Y name: bnxt_en vermagic: 4.19.0+1 SMP mod_unload modversions# dmesg|grep bnxt_en [ 15.047312] Broadcom NetXtreme-C/E/S driver bnxt_en v1.10.3-231.0.162.0 [ 15.048046] bnxt_en 0001:5c:00.0 (unnamed net_device) (uninitialized): Device requests max timeout of 60 seconds, may trigger hung task watchdog [ 15.215453] bnxt_en 0001:5c:00.0 eth0: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff010000, node addr 40:5b:7f:6e:22:00 [ 15.215469] bnxt_en 0001:5c:00.0: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link) [ 15.216606] bnxt_en 0001:5c:00.1 (unnamed net_device) (uninitialized): Device requests max timeout of 60 seconds, may trigger hung task watchdog [ 15.265361] bnxt_en 0001:5c:00.1 eth1: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff000000, node addr 40:5b:7f:6e:22:01 [ 15.265372] bnxt_en 0001:5c:00.1: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link) [ 15.350702] bnxt_en 0001:5c:00.1 side-5183-eth1: renamed from eth1 [ 15.385121] bnxt_en 0001:5c:00.0 side-3345-eth0: renamed from eth0 [ 17.400142] bnxt_en 0001:5c:00.0 eth0: renamed from side-3345-eth0 [ 17.493640] bnxt_en 0001:5c:00.1 eth1: renamed from side-5183-eth1 [ 20.614447] bnxt_en 0001:5c:00.1 eth1: NIC Link is Up, 100000 Mbps (NRZ) full duplex, Flow control: none [ 20.614454] bnxt_en 0001:5c:00.1 eth1: FEC autoneg off encoding: None [ 21.113687] bnxt_en 0001:5c:00.0 eth0: NIC Link is Up, 100000 Mbps (NRZ) full duplex, Flow control: ON - receive & transmit [ 21.113694] bnxt_en 0001:5c:00.0 eth0: FEC autoneg off encoding: None -
RE: Test results for Dell Poweredge R770 with NVMe drives
Relevant lines in 8.2.1 where it succeeds:
[ 12.242658] Broadcom NetXtreme-C/E driver bnxt_en v1.10.0-216.0.119.1
[ 12.274753] bnxt_en 0001:5c:00.0 eth0: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff010000, node addr 40:5b:7f:6e:22:00
[ 12.274767] bnxt_en 0001:5c:00.0: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
[ 12.306588] bnxt_en 0001:5c:00.1 eth1: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff000000, node addr 40:5b:7f:6e:22:01
[ 12.306599] bnxt_en 0001:5c:00.1: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
[ 15.100023] bnxt_en 0001:5c:00.1 side-1659-eth1: renamed from eth1
[ 15.117296] bnxt_en 0001:5c:00.0 side-3513-eth0: renamed from eth0
[ 16.878049] bnxt_en 0001:5c:00.0 eth0: renamed from side-3513-eth0
[ 16.970941] bnxt_en 0001:5c:00.1 eth1: renamed from side-1659-eth1
[ 20.216480] bnxt_en 0001:5c:00.1 eth1: NIC Link is Up, 100000 Mbps full duplex, Flow control: ON - receive & transmit
[ 20.216487] bnxt_en 0001:5c:00.1 eth1: FEC autoneg off encodings: None
[ 20.734402] bnxt_en 0001:5c:00.0 eth0: NIC Link is Up, 100000 Mbps full duplex, Flow control: none
[ 20.734409] bnxt_en 0001:5c:00.0 eth0: FEC autoneg off encodings: None -
RE: Test results for Dell Poweredge R770 with NVMe drives
@olivierlambert Yes, the NIC firmware is the latest.
Strange that it works with 8.2.1 -
RE: XCP-ng 8.3 and Dell R660 - crash during boot, halts remainder of installer process (bnxt_en?)
It still fails with latest iso (xcp-ng-8.3.0-20250606.2.iso) on Dell R770 and Broadcom 57508 100GbE QSFP56 Dual Port Adapter, PCIe Full Height
See post here
By the way, it works ok with 8.2.1 iso -
RE: Test results for Dell Poweredge R770 with NVMe drives
Tested the newest iso (xcp-ng-8.3.0-20250606.2.iso) with Dell R770 and Broadcom 57508 100GbE QSFP56 Dual Port Adapter, PCIe Full Height
It still fails to recognize the NIC, relevant log lines:
[ 57.592461] Broadcom NetXtreme-C/E/S driver bnxt_en v1.10.3-232.0.155.5+
[ 61.400879] bnxt_en 0001:5c:00.0 (unnamed net_device) (uninitialized): Firmware not responding, rc: -16 status: 0x8000
[ 61.421312] bnxt_en: probe of 0001:5c:00.0 failed with error -16By the way, it works ok with 8.2.1 iso
-
RE: LargeBlockSR for 4KiB blocksize disks
Hi, thank you for the response.
Currently these commands return data immediately
[10:52 a1 ~]# vgs WARNING: Not using device /dev/sda for PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94. WARNING: PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94 prefers device /dev/sda.512 because device is used by LV. VG #PV #LV #SN Attr VSize VFree VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679 1 1 0 wz--n- 405.55g 405.55g XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b 1 1 0 wz--n- 27.94t 0 [10:40 a1 ~]# /sbin/vgs --readonly VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679 WARNING: Not using device /dev/sda for PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94. WARNING: PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94 prefers device /dev/sda.512 because device is used by LV. VG #PV #LV #SN Attr VSize VFree VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679 1 1 0 wz--n- 405.55g 405.55g [11:02 a1 log]# /sbin/vgs --readonly XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b WARNING: Not using device /dev/sda for PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94. WARNING: PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94 prefers device /dev/sda.512 because device is used by LV. VG #PV #LV #SN Attr VSize VFree XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b 1 1 0 wz--n- 27.94t 0Maybe this call was somehow in a locked state during the 512 SR create
May 2 08:31:40 a1 SM: [18985] ['/sbin/vgs', '--readonly', 'VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679'] May 2 08:32:24 a1 SM: [18985] pread SUCCESS May 2 08:32:24 a1 SM: [18985] ***** Long LVM call of 'vgs' took 43.6255850792GC seems to be normal for on next attempts
SR:
[11:02 a1 log]# xe sr-list uuid=42535e39-4c98-22c6-71eb-303caa3fc97b uuid ( RO) : 42535e39-4c98-22c6-71eb-303caa3fc97b name-label ( RW): Local Main Nvme storage name-description ( RW): host ( RO): A1 - L 11 - 8.2 - 6740E - 30TB - 512GB - 100G - 2U type ( RO): largeblock content-type ( RO):GC:
May 19 10:40:29 a1 SM: [4167] Kicking GC May 19 10:40:29 a1 SMGC: [4167] === SR 42535e39-4c98-22c6-71eb-303caa3fc97b: gc === May 19 10:40:29 a1 SMGC: [4286] Will finish as PID [4287] May 19 10:40:29 a1 SM: [4287] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 19 10:40:29 a1 SM: [4287] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active May 19 10:40:29 a1 SMGC: [4167] New PID [4286] May 19 10:40:29 a1 SM: [4176] pread SUCCESS May 19 10:40:29 a1 SM: [4176] lock: released /var/lock/sm/.nil/lvm May 19 10:40:29 a1 SM: [4176] lock: released /var/lock/sm/07ab18c4-a76f-d1fc-4374-babfe21fd679/sr May 19 10:40:29 a1 SM: [4176] Entering _checkMetadataVolume May 19 10:40:29 a1 SM: [4176] lock: acquired /var/lock/sm/07ab18c4-a76f-d1fc-4374-babfe21fd679/sr May 19 10:40:29 a1 SM: [4176] sr_scan {'sr_uuid': '07ab18c4-a76f-d1fc-4374-babfe21fd679', 'subtask_of': 'DummyRef:|37b48251-6927-4828-8c6b-38f9b21bc157|SR.scan', 'args': [], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:970bc095-e2eb-434b-bc9f-1c4e8f58b5c2', 'device_config': {'device': '/dev/disk/by-id/nvme-Dell_BOSS-N1_CN0CMFVPFCP0048500D5-part3', 'SRmaster': 'true'}, 'command': 'sr_scan', 'sr_ref': 'OpaqueRef:c67721e5-74b8-47fe-a180-de50665948c5'} May 19 10:40:29 a1 SM: [4176] LVHDSR.scan for 07ab18c4-a76f-d1fc-4374-babfe21fd679 May 19 10:40:29 a1 SM: [4176] lock: acquired /var/lock/sm/.nil/lvm May 19 10:40:29 a1 SM: [4176] ['/sbin/vgs', '--noheadings', '--nosuffix', '--units', 'b', 'VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679'] May 19 10:40:29 a1 SM: [4167] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 19 10:40:29 a1 SM: [4287] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 19 10:40:29 a1 SMGC: [4287] Found 0 cache files May 19 10:40:29 a1 SM: [4287] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active, acquired: True (exists: True) May 19 10:40:29 a1 SM: [4287] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr, acquired: True (exists: True) May 19 10:40:29 a1 SM: [4287] ['/usr/bin/vhd-util', 'scan', '-f', '-m', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b/*.vhd'] May 19 10:40:29 a1 SM: [4287] pread SUCCESS May 19 10:40:29 a1 SMGC: [4287] SR 4253 ('Local Main Nvme storage') (5 VDIs in 3 VHD trees): May 19 10:40:29 a1 SMGC: [4287] 255478b4(20.000G/2.603G) May 19 10:40:29 a1 SMGC: [4287] *5b3775a4(20.000G/2.603G) May 19 10:40:29 a1 SMGC: [4287] fefe7bde(20.000G/881.762M) May 19 10:40:29 a1 SMGC: [4287] 761e5fa7(20.000G/45.500K) May 19 10:40:29 a1 SMGC: [4287] a3f4b8e5(20.000G/2.675G) May 19 10:40:29 a1 SMGC: [4287] May 19 10:40:29 a1 SM: [4287] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 19 10:40:29 a1 SM: [4176] pread SUCCESS May 19 10:40:29 a1 SM: [4176] lock: released /var/lock/sm/.nil/lvm May 19 10:40:29 a1 SMGC: [4287] Got sm-config for *5b3775a4(20.000G/2.603G): {'vhd-blocks': 'eJz7///3DgYgaGBABZTyiQX/oYBc/bjsJ9Y8mNXUcgepAGTnHwb624uwvwHujgHy/z8Ghg9kpx8K7MUK6OUOXPbTyj3o5qDbQC93QNI7kn0HKDWRsYESzQC3eq69'} May 19 10:40:29 a1 SMGC: [4287] No work, exiting May 19 10:40:29 a1 SMGC: [4287] GC process exiting, no work left -
RE: LargeBlockSR for 4KiB blocksize disks
NVMe drives attached to the PERC H965i are identified as SCSI disks in the operating system
OS exposes a 4kn nvme device on /dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05Below is a log of creating a type=largeblock SR on latest XCP-ng 8.2
It takes about 10 minutes and some errors but it does successfully create the SR and we are able to use it.
Are all these errors expected and we can trust it's working normally?Console:
# xe sr-create host-uuid=383399d1-b304-48db-ad4b-bc8fe8b56f89 type=largeblock name-label="Local Main Name storage" device-config:device=/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05 42535e39-4c98-22c6-71eb-303caa3fc97bSM.log
May 2 08:22:29 a1 SM: [15928] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:22:29 a1 SM: [15928] lock: acquired /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:22:29 a1 SM: [15928] sr_create {'sr_uuid': '42535e39-4c98-22c6-71eb-303caa3fc97b', 'subtask_of': 'DummyRef:|2d81471d-e02c-4c9c-8273-51527e849c1d|SR.create', 'args': ['0'], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:61307cea-d759-4f9a-9052-e44c0c574b1f', 'device_config': {'device': '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05', 'SRmaster': 'true'}, 'command': 'sr_create', 'sr_ref': 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013'} May 2 08:22:29 a1 SM: [15928] ['blockdev', '--getss', '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] ['losetup', '-f', '-v', '--show', '--sector-size', '512', '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] util.test_scsiserial: Not a serial device: /dev/loop0 May 2 08:22:29 a1 SM: [15928] lock: opening lock file /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] ['/sbin/vgs', '--readonly', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:22:29 a1 SM: [15928] FAILED in util.pread: (rc 5) stdout: '', stderr: ' WARNING: Not using device /dev/sda for PV HMeziz-gDTa-cNLl-1B2E-ebbh-e5ki-RIhK5T. May 2 08:22:29 a1 SM: [15928] WARNING: PV HMeziz-gDTa-cNLl-1B2E-ebbh-e5ki-RIhK5T prefers device /dev/loop0 because device was seen first. May 2 08:22:29 a1 SM: [15928] Volume group "XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b" not found May 2 08:22:29 a1 SM: [15928] Cannot process volume group XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b May 2 08:22:29 a1 SM: [15928] ' May 2 08:22:29 a1 SM: [15928] ['/bin/dd', 'if=/dev/zero', 'of=/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05.512', 'bs=1M', 'count=10', 'oflag=direct'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] lock: acquired /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] ['/sbin/vgcreate', '--metadatasize', '10M', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b', '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05.512'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] lock: released /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] lock: acquired /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] ['/sbin/vgchange', '-an', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] lock: released /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] lock: acquired /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] ['/sbin/lvdisplay', '/dev/XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:22:29 a1 SM: [15928] FAILED in util.pread: (rc 5) stdout: '', stderr: ' WARNING: Not using device /dev/sda for PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94. May 2 08:22:29 a1 SM: [15928] WARNING: PV tJUoM9-WUfs-XYZy-vXje-p040-30ui-dder94 prefers device /dev/loop0 because device was seen first. May 2 08:22:29 a1 SM: [15928] Failed to find logical volume "XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b" May 2 08:22:29 a1 SM: [15928] ' May 2 08:22:29 a1 SM: [15928] lock: released /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] lock: acquired /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] ['/sbin/vgs', '--noheadings', '--nosuffix', '--units', 'b', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] lock: released /var/lock/sm/.nil/lvm May 2 08:22:29 a1 SM: [15928] ['lvcreate', '-n', '42535e39-4c98-22c6-71eb-303caa3fc97b', '-L', '29302004', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] ['lvchange', '-ay', '/dev/XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:22:29 a1 SM: [15928] pread SUCCESS May 2 08:22:29 a1 SM: [15928] ['mkfs.ext4', '-F', '/dev/XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:31:40 a1 SM: [18985] lock: opening lock file /var/lock/sm/07ab18c4-a76f-d1fc-4374-babfe21fd679/sr May 2 08:31:40 a1 SM: [18985] LVMCache created for VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679 May 2 08:31:40 a1 SM: [18985] lock: opening lock file /var/lock/sm/.nil/lvm May 2 08:31:40 a1 SM: [18985] ['/sbin/vgs', '--readonly', 'VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679'] May 2 08:32:24 a1 SM: [18985] pread SUCCESS May 2 08:32:24 a1 SM: [15928] pread SUCCESS May 2 08:32:24 a1 SM: [15928] ['/usr/lib/udev/scsi_id', '-g', '--device', '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05.512'] May 2 08:32:24 a1 SM: [18985] ***** Long LVM call of 'vgs' took 43.6255850792 May 2 08:32:24 a1 SM: [18985] Entering _checkMetadataVolume May 2 08:32:24 a1 SM: [18985] LVMCache: will initialize now May 2 08:32:24 a1 SM: [18985] LVMCache: refreshing May 2 08:32:24 a1 SM: [18985] lock: acquired /var/lock/sm/.nil/lvm May 2 08:32:24 a1 SM: [18985] ['/sbin/lvs', '--noheadings', '--units', 'b', '-o', '+lv_tags', '/dev/VG_XenStorage-07ab18c4-a76f-d1fc-4374-babfe21fd679'] May 2 08:32:24 a1 SM: [15928] FAILED in util.pread: (rc 1) stdout: '', stderr: '' May 2 08:32:24 a1 SM: [15928] ['losetup', '--list'] May 2 08:32:24 a1 SM: [15928] pread SUCCESS May 2 08:32:24 a1 SM: [15928] ['losetup', '-d', '/dev/loop0'] May 2 08:32:24 a1 SM: [18985] pread SUCCESS May 2 08:32:24 a1 SM: [18985] lock: released /var/lock/sm/.nil/lvm May 2 08:32:24 a1 SM: [15928] pread SUCCESS May 2 08:32:24 a1 SM: [15928] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:24 a1 SM: [19294] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:24 a1 SM: [19294] lock: acquired /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:24 a1 SM: [19294] sr_attach {'sr_uuid': '42535e39-4c98-22c6-71eb-303caa3fc97b', 'subtask_of': 'DummyRef:|ab5bca7f-6597-4874-948a-b4c8a0b4283e|SR.attach', 'args': [], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:5abf6e38-a9b0-44ff-a095-e95786bb30f7', 'device_config': {'device': '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05', 'SRmaster': 'true'}, 'command': 'sr_attach', 'sr_ref': 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013'} May 2 08:32:24 a1 SMGC: [19294] === SR 42535e39-4c98-22c6-71eb-303caa3fc97b: abort === May 2 08:32:24 a1 SM: [19294] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:24 a1 SM: [19294] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active May 2 08:32:24 a1 SM: [19294] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active, acquired: True (exists: True) May 2 08:32:24 a1 SMGC: [19294] abort: releasing the process lock May 2 08:32:24 a1 SM: [19294] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active May 2 08:32:24 a1 SM: [19294] lock: acquired /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:24 a1 SM: [19294] RESET for SR 42535e39-4c98-22c6-71eb-303caa3fc97b (master: True) May 2 08:32:24 a1 SM: [19294] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:24 a1 SM: [19294] set_dirty 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013' succeeded May 2 08:32:24 a1 SM: [19294] ['vgs', '--noheadings', '-o', 'vg_name,devices', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['losetup', '--list'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['losetup', '-f', '-v', '--show', '--sector-size', '512', '/dev/sda'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['vgs', '--noheadings', '-o', 'vg_name,devices', 'XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['lvchange', '-ay', '/dev/XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['fsck', '-a', '/dev/XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['mount', '/dev/XSLocalLargeBlock-42535e39-4c98-22c6-71eb-303caa3fc97b/42535e39-4c98-22c6-71eb-303caa3fc97b', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['/usr/lib/udev/scsi_id', '-g', '--device', '/dev/sda.512'] May 2 08:32:24 a1 SM: [19294] FAILED in util.pread: (rc 1) stdout: '', stderr: '' May 2 08:32:24 a1 SM: [19294] Dom0 disks: ['/dev/nvme0n1p'] May 2 08:32:24 a1 SM: [19294] Block scheduler: /dev/sda.512 (/dev/loop) wants noop May 2 08:32:24 a1 SM: [19294] no path /sys/block/loop/queue/scheduler May 2 08:32:24 a1 SM: [19294] ['/usr/bin/vhd-util', 'scan', '-f', '-m', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b/*.vhd'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] ['ls', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b', '-1', '--color=never'] May 2 08:32:24 a1 SM: [19294] pread SUCCESS May 2 08:32:24 a1 SM: [19294] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running, acquired: True (exists: True) May 2 08:32:24 a1 SM: [19294] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:24 a1 SM: [19294] Kicking GC May 2 08:32:24 a1 SMGC: [19294] === SR 42535e39-4c98-22c6-71eb-303caa3fc97b: gc === May 2 08:32:24 a1 SMGC: [19335] Will finish as PID [19336] May 2 08:32:24 a1 SMGC: [19294] New PID [19335] May 2 08:32:24 a1 SM: [19294] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19336] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SMGC: [19336] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* May 2 08:32:25 a1 SMGC: [19336] *********************** May 2 08:32:25 a1 SMGC: [19336] * E X C E P T I O N * May 2 08:32:25 a1 SMGC: [19336] *********************** May 2 08:32:25 a1 SMGC: [19336] gc: EXCEPTION <class 'util.SMException'>, SR 42535e39-4c98-22c6-71eb-303caa3fc97b not attached on this host May 2 08:32:25 a1 SMGC: [19336] File "/opt/xensource/sm/cleanup.py", line 3388, in gc May 2 08:32:25 a1 SMGC: [19336] _gc(None, srUuid, dryRun) May 2 08:32:25 a1 SMGC: [19336] File "/opt/xensource/sm/cleanup.py", line 3267, in _gc May 2 08:32:25 a1 SMGC: [19336] sr = SR.getInstance(srUuid, session) May 2 08:32:25 a1 SMGC: [19336] File "/opt/xensource/sm/cleanup.py", line 1552, in getInstance May 2 08:32:25 a1 SMGC: [19336] return FileSR(uuid, xapi, createLock, force) May 2 08:32:25 a1 SMGC: [19336] File "/opt/xensource/sm/cleanup.py", line 2334, in __init__ May 2 08:32:25 a1 SMGC: [19336] SR.__init__(self, uuid, xapi, createLock, force) May 2 08:32:25 a1 SMGC: [19336] File "/opt/xensource/sm/cleanup.py", line 1582, in __init__ May 2 08:32:25 a1 SMGC: [19336] raise util.SMException("SR %s not attached on this host" % uuid) May 2 08:32:25 a1 SMGC: [19336] May 2 08:32:25 a1 SMGC: [19336] *~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~*~* May 2 08:32:25 a1 SMGC: [19336] * * * * * SR 42535e39-4c98-22c6-71eb-303caa3fc97b: ERROR May 2 08:32:25 a1 SMGC: [19336] May 2 08:32:25 a1 SM: [19367] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19367] sr_update {'sr_uuid': '42535e39-4c98-22c6-71eb-303caa3fc97b', 'subtask_of': 'DummyRef:|f960ef27-5d11-461d-9d4f-072e24be96b0|SR.stat', 'args': [], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:220b9035-d899-4882-9627-bd6d4adb9e9c', 'device_config': {'device': '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05', 'SRmaster': 'true'}, 'command': 'sr_update', 'sr_ref': 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013'} May 2 08:32:25 a1 SM: [19387] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19387] lock: acquired /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19387] sr_scan {'sr_uuid': '42535e39-4c98-22c6-71eb-303caa3fc97b', 'subtask_of': 'DummyRef:|9e3a3942-fb33-46d2-bb01-5463e16ff9a1|SR.scan', 'args': [], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:7e309bfe-889c-464e-b6ed-949d9e4adfb5', 'device_config': {'device': '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05', 'SRmaster': 'true'}, 'command': 'sr_scan', 'sr_ref': 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013'} May 2 08:32:25 a1 SM: [19387] ['/usr/bin/vhd-util', 'scan', '-f', '-m', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b/*.vhd'] May 2 08:32:25 a1 SM: [19387] pread SUCCESS May 2 08:32:25 a1 SM: [19387] ['ls', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b', '-1', '--color=never'] May 2 08:32:25 a1 SM: [19387] pread SUCCESS May 2 08:32:25 a1 SM: [19387] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:25 a1 SM: [19387] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running, acquired: True (exists: True) May 2 08:32:25 a1 SM: [19387] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:25 a1 SM: [19387] Kicking GC May 2 08:32:25 a1 SMGC: [19387] === SR 42535e39-4c98-22c6-71eb-303caa3fc97b: gc === May 2 08:32:25 a1 SMGC: [19398] Will finish as PID [19399] May 2 08:32:25 a1 SM: [19399] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/running May 2 08:32:25 a1 SM: [19399] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active May 2 08:32:25 a1 SMGC: [19387] New PID [19398] May 2 08:32:25 a1 SM: [19387] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19399] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SMGC: [19399] Found 0 cache files May 2 08:32:25 a1 SM: [19399] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active, acquired: True (exists: True) May 2 08:32:25 a1 SM: [19399] lock: tried lock /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr, acquired: True (exists: True) May 2 08:32:25 a1 SM: [19399] ['/usr/bin/vhd-util', 'scan', '-f', '-m', '/var/run/sr-mount/42535e39-4c98-22c6-71eb-303caa3fc97b/*.vhd'] May 2 08:32:25 a1 SM: [19399] pread SUCCESS May 2 08:32:25 a1 SMGC: [19399] SR 4253 ('Local Main Name storage') (0 VDIs in 0 VHD trees): no changes May 2 08:32:25 a1 SM: [19399] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SMGC: [19399] No work, exiting May 2 08:32:25 a1 SMGC: [19399] GC process exiting, no work left May 2 08:32:25 a1 SM: [19399] lock: released /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/gc_active May 2 08:32:25 a1 SMGC: [19399] In cleanup May 2 08:32:25 a1 SMGC: [19399] SR 4253 ('Local Main Name storage') (0 VDIs in 0 VHD trees): no changes May 2 08:32:25 a1 SM: [19432] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19432] sr_update {'sr_uuid': '42535e39-4c98-22c6-71eb-303caa3fc97b', 'subtask_of': 'DummyRef:|3207059a-03f4-42a3-bd23-23d226386b08|SR.stat', 'args': [], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:1a441cae-d761-45b0-a025-c2ca371f0639', 'device_config': {'device': '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05', 'SRmaster': 'true'}, 'command': 'sr_update', 'sr_ref': 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013'} May 2 08:32:25 a1 SM: [19449] lock: opening lock file /var/lock/sm/42535e39-4c98-22c6-71eb-303caa3fc97b/sr May 2 08:32:25 a1 SM: [19449] sr_update {'sr_uuid': '42535e39-4c98-22c6-71eb-303caa3fc97b', 'subtask_of': 'DummyRef:|6486f365-f80b-427e-afc5-e5c8cc1b4931|SR.stat', 'args': [], 'host_ref': 'OpaqueRef:a26b109a-5fb2-4644-8e7e-3cc251e43d5c', 'session_ref': 'OpaqueRef:1501190d-78ae-4d34-9b7f-bc6fb1103494', 'device_config': {'device': '/dev/disk/by-id/scsi-36f4ee0807844d90068121b9321711e05', 'SRmaster': 'true'}, 'command': 'sr_update', 'sr_ref': 'OpaqueRef:339f5ed6-e8e6-4421-932f-d64f7ef60013'} -
Test results for Dell Poweredge R770 with NVMe drives
I will share some test results for the community, maybe helps somebody.
1. Server spec:
This is the latest generation Dell server with Intel Efficiency core CPUs and NVMe disks.
All firmware and BIOS versions are latest as of 28.04.2025Dell Poweredge R770
2.5" Chassis with up to 8 NVMe HWRAID Drives, Front PERC 12 (H965i)
2 x Intel
Xeon
6 Efficient 6740E 2.4G, 96C/96T, 24GT/s, 96M Cache, Turbo, (250W) DDR5-6400
BOSS-N1 controller card + with 2 M.2 480GB (RAID 1) (22x80) Rear
Broadcom 57508 Dual Port 100GbE QSFP56, OCP NIC 3.0
2 x 800G Data Center NVMe Mixed Use AG Drive U2 Gen4 with carrier
2 x 32GB RDIMM, 6400MT/s, Dual RankVirtual disks set up in iDRAC:
vd1 - RAID1 (2 x 480GB) virtual disk configured in iDRAC
vd2 - RAID1 (2 x 800G) virtual disk configured in Perc H965i2. Results for latest XCP-ng 8.2
Virtual disks and network is detected normally.
XCP-ng installation to vd1 is successful.
Since vd2 has 4K sectors, it is not usable in a standard way.
Need to use SR with type=largeblock (https://xcp-ng.org/forum/topic/8901/largeblocksr-for-4kib-blocksize-disks).
Currently only one SR with type=largeblock can be created in one host. Adding multiple will cause errors.
In the long run there will be support for 4k sector NVMe disks with SMAPIv3 (https://xcp-ng.org/blog/2024/04/19/first-smapiv3-driver-is-available-in-preview/)3. Results for latest XCP-ng 8.3
Never reaches install console
Fails to detect storage and network
Gets stuck at "Started Update UTMP about System Runlevel Changes." as seen on screenshot.
I did try starting install console with command "/opt/xensource/installer/preinit console" and it started but cant get too far with it, see last 2 screenshots






-
RE: LargeBlockSR for 4KiB blocksize disks
@dthenot Are multiple SR-s with type=largeblock supported on the same host?