XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. timewasted
    3. Posts
    T
    • Profile
    • Following 0
    • Followers 0
    • Topics 0
    • Posts 2
    • Best 1
    • Controversial 0
    • Groups 0

    Posts made by timewasted

    • RE: Epyc VM to VM networking slow

      Just out of curiosity, how is everyone that's experiencing this issue currently dealing with it while the issue is being investigated? I was sort of naively hoping that it would get sorted by the 8.3 release, but now that those hopes have been dashed I'm trying to see what options I have to work around the issue.

      posted in Compute
      T
      timewasted
    • RE: Epyc VM to VM networking slow

      @bleader I've been investigating this issue on my own system and came across this discussion. I know this is a somewhat old thread so I hope it's ok to contribute more data here!

      Host:

      • CPU: EPYC 7302p
      • Number of sockets: 1
      • CPU pinning: no
      • XCP-NG version: 8.3 beta 2, Xen 4.17 (everything current as of the time of writing)
      • Output of xl info -n:
      host                   : xcp-ng
release                : 4.19.0+1
version                : #1 SMP Wed Jan 24 17:19:11 CET 2024
machine                : x86_64
nr_cpus                : 32
max_cpu_id             : 31
nr_nodes               : 1
cores_per_socket       : 16
threads_per_core       : 2
cpu_mhz                : 3000.001
hw_caps                : 178bf3ff:7ed8320b:2e500800:244037ff:0000000f:219c91a9:00400004:00000780
virt_caps              : pv hvm hvm_directio pv_directio hap gnttab-v1 gnttab-v2
total_memory           : 130931
free_memory            : 24740
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
cpu_topology           :
cpu:    core    socket     node
  0:       0        0        0
  1:       0        0        0
  2:       1        0        0
  3:       1        0        0
  4:       4        0        0
  5:       4        0        0
  6:       5        0        0
  7:       5        0        0
  8:       8        0        0
  9:       8        0        0
 10:       9        0        0
 11:       9        0        0
 12:      12        0        0
 13:      12        0        0
 14:      13        0        0
 15:      13        0        0
 16:      16        0        0
 17:      16        0        0
 18:      17        0        0
 19:      17        0        0
 20:      20        0        0
 21:      20        0        0
 22:      21        0        0
 23:      21        0        0
 24:      24        0        0
 25:      24        0        0
 26:      25        0        0
 27:      25        0        0
 28:      28        0        0
 29:      28        0        0
 30:      29        0        0
 31:      29        0        0
device topology        :
device           node
No device topology data available
numa_info              :
node:    memsize    memfree    distances
   0:    132338      24740      10
xen_major              : 4
xen_minor              : 17
xen_extra              : .3-3
xen_version            : 4.17.3-3
xen_caps               : xen-3.0-x86_64 hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : $Format:%H$, pq ???
xen_commandline        : dom0_mem=7568M,max:7568M watchdog ucode=scan dom0_max_vcpus=1-16 crashkernel=256M,below=4G console=vga vga=mode-0x0311
cc_compiler            : gcc (GCC) 11.2.1 20210728 (Red Hat 11.2.1-1)
cc_compile_by          : mockbuild
cc_compile_domain      : [unknown]
cc_compile_date        : Wed Feb 28 10:12:19 CET 2024
build_id               : 9a011a28e29a21a7643376b36aec959253587d42
xend_config_format     : 4

      Test set 1:

      Server and client were both Debian 12 (Linux 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64) with 4 cores.

      VM to VM (1 thread):

      iperf -c 192.168.1.66 -t 60
------------------------------------------------------------
Client connecting to 192.168.1.66, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  1] local 192.168.1.69 port 55530 connected with 192.168.1.66 port 5001 (icwnd/mss/irtt=14/1448/551)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-60.0213 sec  38.2 GBytes  5.47 Gbits/sec

xentop: 100 / 150 / 250

      VM to VM (4 threads):

      iperf -c 192.168.1.66 -t 60 -P4
------------------------------------------------------------
Client connecting to 192.168.1.66, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  2] local 192.168.1.69 port 35702 connected with 192.168.1.66 port 5001 (icwnd/mss/irtt=14/1448/531)
[  4] local 192.168.1.69 port 35708 connected with 192.168.1.66 port 5001 (icwnd/mss/irtt=14/1448/576)
[  1] local 192.168.1.69 port 35714 connected with 192.168.1.66 port 5001 (icwnd/mss/irtt=14/1448/458)
[  3] local 192.168.1.69 port 35692 connected with 192.168.1.66 port 5001 (icwnd/mss/irtt=14/1448/744)
[ ID] Interval       Transfer     Bandwidth
[  2] 0.0000-60.0141 sec  12.4 GBytes  1.77 Gbits/sec
[  1] 0.0000-60.0129 sec  13.9 GBytes  1.99 Gbits/sec
[  3] 0.0000-60.0141 sec  14.5 GBytes  2.07 Gbits/sec
[  4] 0.0000-60.0301 sec  12.2 GBytes  1.75 Gbits/sec
[SUM] 0.0000-60.0071 sec  53.0 GBytes  7.58 Gbits/sec

xentop: 165 / 200 / 380

      Host to VM (1 thread):

      iperf -c 192.168.1.66 -t 60
------------------------------------------------------------
Client connecting to 192.168.1.66, TCP port 5001
TCP window size:  297 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.1 port 37804 connected with 192.168.1.66 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  6.58 GBytes   942 Mbits/sec

xentop: N/A / 135 / 145

      Host to VM (4 threads):

      iperf -c 192.168.1.66 -t 60 -P4
------------------------------------------------------------
Client connecting to 192.168.1.66, TCP port 5001
TCP window size:  112 KByte (default)
------------------------------------------------------------
[  5] local 192.168.1.1 port 37812 connected with 192.168.1.66 port 5001
[  3] local 192.168.1.1 port 37808 connected with 192.168.1.66 port 5001
[  6] local 192.168.1.1 port 37814 connected with 192.168.1.66 port 5001
[  4] local 192.168.1.1 port 37810 connected with 192.168.1.66 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  1.63 GBytes   233 Mbits/sec
[  6]  0.0-60.0 sec  2.08 GBytes   298 Mbits/sec
[  4]  0.0-60.0 sec  1.07 GBytes   154 Mbits/sec
[  5]  0.0-60.0 sec  1.80 GBytes   257 Mbits/sec
[SUM]  0.0-60.0 sec  6.58 GBytes   942 Mbits/sec

xentop: N/A / 155 / 155

      Test set 2:

      Server: FreeBSD 14 (FreeBSD 14.0-RELEASE (GENERIC) #0 releng/14.0-n265380-f9716eee8ab4: Fri Nov 10 05:57:23 UTC 2023) with 4 cores.
      Client: Debian 12 (Linux 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64) with 4 cores.

      VM to VM (1 thread):

      iperf -c 192.168.1.64 -t 60
------------------------------------------------------------
Client connecting to 192.168.1.64, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  1] local 192.168.1.69 port 38572 connected with 192.168.1.64 port 5001 (icwnd/mss/irtt=14/1448/905)
[ ID] Interval       Transfer     Bandwidth
[  1] 0.0000-60.0089 sec  21.3 GBytes  3.04 Gbits/sec

xentop: 125 / 355 / 325

      VM to VM (4 threads):

      iperf -c 192.168.1.64 -t 60 -P4
------------------------------------------------------------
Client connecting to 192.168.1.64, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.69 port 50068 connected with 192.168.1.64 port 5001 (icwnd/mss/irtt=14/1448/753)
[  1] local 192.168.1.69 port 50078 connected with 192.168.1.64 port 5001 (icwnd/mss/irtt=14/1448/513)
[  4] local 192.168.1.69 port 50088 connected with 192.168.1.64 port 5001 (icwnd/mss/irtt=14/1448/411)
[  2] local 192.168.1.69 port 50070 connected with 192.168.1.64 port 5001 (icwnd/mss/irtt=14/1448/676)
[ ID] Interval       Transfer     Bandwidth
[  4] 0.0000-60.0299 sec  9.48 GBytes  1.36 Gbits/sec
[  1] 0.0000-60.0299 sec  6.56 GBytes   938 Mbits/sec
[  3] 0.0000-60.0301 sec  11.2 GBytes  1.60 Gbits/sec
[  2] 0.0000-60.0293 sec  6.61 GBytes   947 Mbits/sec
[SUM] 0.0000-60.0146 sec  33.8 GBytes  4.84 Gbits/sec

xentop: 220 / 400 / 730

      Host to VM (1 thread):

      iperf -c 192.168.1.64 -t 60
------------------------------------------------------------
Client connecting to 192.168.1.64, TCP port 5001
TCP window size:  212 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.1 port 58464 connected with 192.168.1.64 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-60.0 sec  6.58 GBytes   941 Mbits/sec

xentop: N/A / 295 / 205

      Host to VM (4 threads):

      iperf -c 192.168.1.64 -t 60 -P4
------------------------------------------------------------
Client connecting to 192.168.1.64, TCP port 5001
TCP window size:  130 KByte (default)
------------------------------------------------------------
[  5] local 192.168.1.1 port 58470 connected with 192.168.1.64 port 5001
[  3] local 192.168.1.1 port 58468 connected with 192.168.1.64 port 5001
[  4] local 192.168.1.1 port 58472 connected with 192.168.1.64 port 5001
[  6] local 192.168.1.1 port 58474 connected with 192.168.1.64 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-60.0 sec  1.73 GBytes   247 Mbits/sec
[  3]  0.0-60.0 sec  1.56 GBytes   224 Mbits/sec
[  4]  0.0-60.0 sec  1.73 GBytes   247 Mbits/sec
[  6]  0.0-60.0 sec  1.56 GBytes   224 Mbits/sec
[SUM]  0.0-60.0 sec  6.58 GBytes   942 Mbits/sec

xentop: N/A / 280 / 205

      Conclusion:

      No special tuning on any of the VMs, just a fresh install from the netboot ISO for each respective OS.

      I also don't fully understand why my host seems to be limited to 1Gb. The management interface is 1Gb, but that shouldn't matter? The other physical NIC is 10Gb SFP+, just for the sake of completeness.

      Please let me know if there's anything at all that I can do to help with this!

      posted in Compute
      T
      timewasted