Xen ERMS Patch - Call for performance testing

TeddyAstie

Hello !

I am looking to get some feedback and evaluation on a performance-related patch for Xen (XCP-ng 8.3 only).
This patch changes the memcpy implementation of Xen to use the "ERMS variant" (aka REP MOVSB) instead of the current REP MOVSQ+B implementation.
This is expected to perform better on the vast majority of Intel CPUs and modern AMD ones (Zen3+), but may perform worse on some older AMD CPUs.

This change may impact the performance of PV drivers (especially network).

You can find more details regarding this proposed change in : https://github.com/xcp-ng-rpms/xen/pull/54
This change may be reworked in the future to take more in account the specificities of each CPUs (e.g check presence of ERMS flag).

Keep in mind that this patched version is experimental and not officially supported.

Installation :

# Download repo file for XCP-ng 8.3
wget https://koji.xcp-ng.org/repos/user/8/8.3/xcpng-users.repo -O /etc/yum.repos.d/xcpng-users.repo

# Installing the patched Xen packages (you should see `.erms` packages)
yum update --enablerepo=xcp-ng-tae1

You can revert the changes by downgrading the Xen package with the ones in the default repos.

yum downgrade --disablerepo=xcp-ng-tae1 "xen-*"

TSnake41 opened this pull request in xcp-ng-rpms/xen

draft Use ERMS variant for memcpy #54

olivierlambert

Might be interesting to give some expected benchmarks like iperf examples with it

TeddyAstie

@olivierlambert

iperf (-P8) for VM to VM on Xeon Gold 6138
Before: 25-35 Gbps
After: 25-39 Gbps

Seems slightly higher at best, but hard to measure a difference as the performance tend to vary a lot between runs.

Forza

Would be very interesting to see the performance on EPYC systems.

Andrew

@TeddyAstie I ran iperf3 tests between two VMs on Xeon E5-2680v2 and average results were within 2% of each other with the patch being slower. Peak speed was 14.5 vs. 13.7 Gbits/sec.