Epyc VM to VM networking slow
-
@nicols We're on 2 HP's with AMD EPYC 7543P 32-Core and it is a pain.
Had to deploy Intel server just for running the backups, which improved a lot (but that's stupid with 2 beasts of hosts) -
No obvious solution yet, it's likely due to an architecture problem on AMD, because of CCDs and how CPUs are made. So the solution (if there's any) will be likely a sum of various small improvements to make it bearable.
I'm going to Santa Clara to discuss that with AMD directly (among other things).
-
@olivierlambert No more EPYC's here, that's for sure. In the future it'll be Intel again (even if they then belong to Broadcom and TMC :p)
-
@olivierlambert said in Epyc VM to VM networking slow:
No obvious solution yet, it's likely due to an architecture problem on AMD, because of CCDs and how CPUs are made. So the solution (if there's any) will be likely a sum of various small improvements to make it bearable.
I'm going to Santa Clara to discuss that with AMD directly (among other things).
Do we have other data to back this? The issue is not really common outside of Xen. I do hope some solution comes out from the meeting with AMD.
-
If we become partners officially, we'll be able to have more advanced accesses with their teams. I still have hope, it's just that the pace isn't on me.
-
@olivierlambert
Can we rule out extra_guest_irqs as the root cause of this problem?https://docs.xcp-ng.org/compute/#nvme-storage-devices-on-linux -
It's probably completely unrelated, but feel free to test
-
@olivierlambert said in Epyc VM to VM networking slow:
If we become partners officially, we'll be able to have more advanced accesses with their teams. I still have hope, it's just that the pace isn't on me.
Hi, is there anything new to report on this? We have very powerful machines, but unfortunately limited by this stubborn issue.
-
@Forza Dito. A 15.000€ EPYC HP monster is slower than a 1.600€ Protectli Intel...
This is a joke and had we known this we'd NEVER jumped on the AMD wagon
-
@Forza said in Epyc VM to VM networking slow:
olivierlambert said in Epyc VM to VM networking slow:
If we become partners officially, we'll be able to have more advanced accesses with their teams. I still have hope, it's just that the pace isn't on me.
Hi, is there anything new to report on this? We have very powerful machines, but unfortunately limited by this stubborn issue.
Can you test https://xcp-ng.org/forum/topic/10862/early-testable-pvh-support ?
We observe very significant improvements on AMD EPYC with PVH.
We're still pin-pointing the issue with HVM, the current hypothesis is a issue regarding memory typing (grant-table accessed as uncacheable(UC) which is very slow) related to grant-table positionning in HVM.
-
@TeddyAstie Unfortunately not. This is a production pool on 8.2.1 so I do not want to try too experimental things.
Do we know if the issue happens on plain Xen on a modern (6.12-15) dom0 kernel?
-
It's on any Xen and Linux version. Vates is now the spearhead on finding the problem and a solution, there's no upstream with a fix anywhere.
