Epyc VM to VM networking slow

ksyblast

@olivierlambert we are using Oracle Linux, is OEL8/9/10 supported? As I understand OEL8 and 9 do contain a fix. We can also try with any other OS suggested by you. Thanks!

olivierlambert

Question for @Team-Hypervisor-Kernel

stormi

OEL 8 & 9 wouldn't contain the fix unless they applied extra patches for this to the RHEL 8 & 9 kernel(s). I'll let the hypervisor team check the current status.

dinhngtu

kernel-4.18.0-553.71.1.0.1.el8_10 (OL8) and kernel-5.14.0-570.37.1.0.1.el9_6 (OL9) do not contain the fix. kernel-6.12.0-55.29.1.0.1.el10_0 (OL10) does.

olivierlambert

Thanks @dinhngtu and @stormi

Forza

Tested the new updates on my prod EPYC 7402P pool with iperf3. Seems like quite a good uplift

Ubuntu 24.04 VM (6 cores) -> bare metal server (6 cores) over a 2x25Gbit LACP link.

Pre-patch

iperf3 -P1 : 9.72Gbit/s
iperf3 -P6 : 14.6GBis/s

Post Patch

iperf3 -P1 : 11.3GBit/s
iperf3 -P6 : 24.2GBit/s

Ubuntu 24.04 VM (6 cores) -> Ubuntu 24.04 VM (6 cores) on the same host

Pre Patch

Forgot to test this...

Post Patch

iperf3 -P1 : 13.7GBit/s
iperf3 -P6 : 30.8GBit/s
iperf3 -P24 : 40.4GBit/s

Our servers have Last-Level Cache (LLC) as NUMA Node enabled as most our VMs do not have huge amount of vCPUs assigned. This means for the EPYC 7402P (24c/48t) we have 8 NUMA nodes. We however do not use xl cpupool-numa-split.

olivierlambert

That's nice! It means it scales relatively well with many threads, it's a good result