Intel x710-t2l Problems

wlp94114

This info may also shed light:

[08:39 xandria /]# dmesg | grep i40e
[   24.513039] i40e: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver - version 2.25.11
[   24.513042] i40e: Copyright (C) 2013-2024 Intel Corporation
[   24.530151] i40e 0000:04:00.0: fw 6.0.48442 api 1.7 nvm 6.01 0x80003484 1.1747.0
[   24.600704] i40e 0000:04:00.0: MAC source pruning enabled on all VFs
[   24.600844] i40e 0000:04:00.0: MAC address: 40:a6:b7:b8:45:e8
[   24.601304] i40e 0000:04:00.0: FW LLDP is disabled
[   24.601361] i40e 0000:04:00.0: FW LLDP is disabled, attempting SW DCB
[   24.609522] i40e 0000:04:00.0: SW DCB initialization succeeded.
[   24.612187] i40e 0000:04:00.0: MAC source pruning enabled on all VFs
[   24.626574] i40e 0000:04:00.0 eth1: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: RX/TX
[   24.628624] i40e 0000:04:00.0: PCI-Express: Speed 8.0GT/s Width x8
[   24.637352] i40e 0000:04:00.0: Features: PF-id[0] VFs: 32 VSIs: 34 QP: 16 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA
[   24.654350] i40e 0000:04:00.1: fw 6.0.48442 api 1.7 nvm 6.01 0x80003484 1.1747.0
[   24.823713] i40e 0000:04:00.1: MAC source pruning enabled on all VFs
[   24.823850] i40e 0000:04:00.1: MAC address: 40:a6:b7:b8:45:e9
[   24.824318] i40e 0000:04:00.1: FW LLDP is disabled
[   24.824375] i40e 0000:04:00.1: FW LLDP is disabled, attempting SW DCB
[   24.832392] i40e 0000:04:00.1: SW DCB initialization succeeded.
[   24.834904] i40e 0000:04:00.1: MAC source pruning enabled on all VFs
[   24.848482] i40e 0000:04:00.1 eth3: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: RX/TX
[   24.850610] i40e 0000:04:00.1: PCI-Express: Speed 8.0GT/s Width x8
[   24.859297] i40e 0000:04:00.1: Features: PF-id[1] VFs: 32 VSIs: 34 QP: 16 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA
[   24.876551] i40e 0000:04:00.2: fw 6.0.48442 api 1.7 nvm 6.01 0x80003484 1.1747.0
[   24.941234] i40e 0000:04:00.2: MAC source pruning enabled on all VFs
[   24.941368] i40e 0000:04:00.2: MAC address: 40:a6:b7:b8:45:ea
[   24.941827] i40e 0000:04:00.2: FW LLDP is disabled
[   24.941883] i40e 0000:04:00.2: FW LLDP is disabled, attempting SW DCB
[   24.949994] i40e 0000:04:00.2: SW DCB initialization succeeded.
[   24.952571] i40e 0000:04:00.2: MAC source pruning enabled on all VFs
[   24.966298] i40e 0000:04:00.2 eth5: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: RX/TX
[   24.968447] i40e 0000:04:00.2: PCI-Express: Speed 8.0GT/s Width x8
[   24.977158] i40e 0000:04:00.2: Features: PF-id[2] VFs: 32 VSIs: 34 QP: 16 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA
[   24.994453] i40e 0000:04:00.3: fw 6.0.48442 api 1.7 nvm 6.01 0x80003484 1.1747.0
[   25.063546] i40e 0000:04:00.3: MAC source pruning enabled on all VFs
[   25.063681] i40e 0000:04:00.3: MAC address: 40:a6:b7:b8:45:eb
[   25.064149] i40e 0000:04:00.3: FW LLDP is disabled
[   25.064205] i40e 0000:04:00.3: FW LLDP is disabled, attempting SW DCB
[   25.072278] i40e 0000:04:00.3: SW DCB initialization succeeded.
[   25.075960] i40e 0000:04:00.3: MAC source pruning enabled on all VFs
[   25.089578] i40e 0000:04:00.3 eth7: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: RX/TX
[   25.091713] i40e 0000:04:00.3: PCI-Express: Speed 8.0GT/s Width x8
[   25.100391] i40e 0000:04:00.3: Features: PF-id[3] VFs: 32 VSIs: 34 QP: 16 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA
[   25.117883] i40e 0000:41:00.0: fw 9.154.78653 api 1.15 nvm 9.54 0x8000fb5f 1.3800.0
[   25.306399] i40e 0000:41:00.0: MAC source pruning enabled on all VFs
[   25.306607] i40e 0000:41:00.0: MAC address: 68:05:ca:df:3b:26
[   25.307055] i40e 0000:41:00.0: FW LLDP is disabled
[   25.307111] i40e 0000:41:00.0: FW LLDP is disabled, attempting SW DCB
[   25.313620] i40e 0000:41:00.0: SW DCB initialization succeeded.
[   25.316748] i40e 0000:41:00.0: MAC source pruning enabled on all VFs
[   25.322371] i40e 0000:41:00.0: PCI-Express: Speed 8.0GT/s Width x8
[   25.324617] i40e 0000:41:00.0: Features: PF-id[0] VFs: 64 VSIs: 66 QP: 16 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA
[   25.342061] i40e 0000:41:00.1: fw 9.154.78653 api 1.15 nvm 9.54 0x8000fb5f 1.3800.0
[   25.402254] i40e 0000:41:00.1: MAC source pruning enabled on all VFs
[   25.402406] i40e 0000:41:00.1: MAC address: 68:05:ca:df:3b:27
[   25.402924] i40e 0000:41:00.1: FW LLDP is disabled
[   25.402981] i40e 0000:41:00.1: FW LLDP is disabled, attempting SW DCB
[   25.409497] i40e 0000:41:00.1: SW DCB initialization succeeded.
[   25.412550] i40e 0000:41:00.1: MAC source pruning enabled on all VFs
[   25.418165] i40e 0000:41:00.1: PCI-Express: Speed 8.0GT/s Width x8
[   25.420417] i40e 0000:41:00.1: Features: PF-id[1] VFs: 64 VSIs: 66 QP: 16 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE P

wlp94114

@olivierlambert

Good morning!

I've performed a clean v8.3 install with all updates.

As before, the Intel x710-DA4 works when added to a vm in Xen Orchestra. I'm still unable to do a pci passthrough to a TrueNAS vm. I can pass LSI hba and Intel datacenter nvmes.

The Intel x710-T2l is listed via lspci but is otherwise unresponsive.

Your thoughts?

olivierlambert

Hi!

What do you mean by "unresponsive"?

wlp94114

@olivierlambert

Good morning!

I've created a Debian 13 vm with x710-T2l as the network device.

When I try to install Debian 13, the installer fails when trying to acquire a DHCP address.

My original plan was/is to build a pfSense router/firewall using the x710. The installer configures the interfaces (WAN/LAN) but fails when trying to access the internet.

The XCP-ng v8.3 host is using the alt i40 driver.

olivierlambert

And if you add a fixed address, does it work?

wlp94114

@olivierlambert

I ran through the install with a fixed address without success.

I've attached some screenshots for your perusal.

olivierlambert

Is your card having a recent firmware? I don't know why a passthrough NIC wouldn't work

splastunov

try to disable tx/rx offload from xcp-ng side.

get VIF list of your VM

xe vm-vif-list uuid=VMUUID

disable features for each VIF

xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"

reboot VM

wlp94114

@olivierlambert
Yes, there has been a recent firmware update.

Here is the info on the x710-T2l:
[08:19 xandria ~]# ethtool -i eth9,
driver: i40e
version: 2.26.8
firmware-version: 9.54 0x8000fb5f 1.3800.0
expansion-rom-version:
bus-info: 0000:41:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

The x710-DA4, which is assignable to a vm via Xen Orchestra but not assignable via pci passthrough, is on a much older firmware rev:
[08:24 xandria ~]# ethtool -i eth4
driver: i40e
version: 2.26.8
firmware-version: 6.01 0x80003484 1.1747.0
expansion-rom-version:
bus-info: 0000:04:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

wlp94114

@splastunov said in Intel x710-t2l Problems:

xe vif-param-set uuid=VIFUUID other-config:ethtool-tx="off"
xe vif-param-set uuid=VIFUUID other-config:ethtool-rx="off"

Hi!

Thanks for your input. I set the vif parameters as suggested and am still unable to ping the gateway.

wlp94114

@olivierlambert

I realize this is an older topic and I'd like to share you with what I have learned.

The x710 series has a reputation for being picky about driver and nvram versions.

According to the "332191_Intel Ethernet Controller X710-XXV710-XL710 Feature Summary_Rev5_12" publication, page 34, the closest match for XCP-ng's intel-i40e-alt.x86_64 (version: 2.26.8) is nvram 9.52.

I flashed both the x710-T2l and the X710-DA4 to 9.52 and have success passing the x710-DA4 through to a vm.

I haven't tried to do a passthrough with the x710-T2l; I've got PfSense running using XCP-ng's vlan feature and some tuneables in PfSense and am reluctant to rock the boat.

I'm including a snapshot of page 34 of the Intel pub for your review.

Best regards!

x710 nvram-driver versions.png

olivierlambert

Thank you very much for sharing your findings

Adding @Team-Documentation-Knowledge-Management and @Team-OS-Platform-Release to check what's next on our side

yann

Adding this to our topics to discuss

yann

@wlp94114 said in Intel x710-t2l Problems:

The x710 series has a reputation for being picky about driver and nvram versions.

Out of curiosity, is there anything reported by the linux driver that would allow make the situation obvious?

wlp94114

@yann Hi yann,

Sorry for the radio silence. Have been away.

How would I check for that?

yann

@wlp94114 for example, did you get any warnings in the kernel logs about the firmware version mismatch?