Offline
@Danp said in Incorret time in bash-prompt and logs on XOA:
The bash prompt may be displaying the incorrect time due to these lines in /etc/bash.bashrc --
# Default user timezone is EST if [ -z "$TZ" ] then export TZ=EST fiAre you getting the expected result if you comment out this section of code and then reboot?
Had the same issue myself, and removing that helped.
@olivierlambert said in XCP-ng Guest Agent - Reported Windows Version for Servers:
It's funny to see Microsoft having a version 10 for an edition named 11. I suppose it's not a surprise for an organization that huge.
They did say that Windows 10 would be the last version of Windows... 
@samuelolavo, nice work! I wonder if it can be improved by using an independent machine/unit and login remotely via ssh and execute the script? This way we need no modification on xcp-ng dom0 itself.
@kagbasi-ngc As far as I know, Linux cannot release it without rebooting.
@kagbasi-ngc A fair warning that lazy unmount doesn't release the mount or any active file descriptors, just hides them from them from the mount output.
@dinhngtu said in Epyc VM to VM networking slow:
@Forza There's a new script here that will help you check the VM's status wrt. the Fix 1.
Thank you. It does indeed look like the EPYC fix is active in XOA.
[07:25 22] xoa:~$ python3 epyc-fix-check.py
'xen-platform-pci' PCI IO mem address is 0xFB000000
Grant table cacheability fix is ACTIVE.
Has Vates checked if a newer kernel would help the network performance with XOA?
Current kernel is: linux-image-amd64/oldstable,now 6.1.148-1 amd64 [installed]
When trying to install any of the newer kernels (6.12.43-*) it immediately fails dependency check:
[07:30 22] xoa:~$ apt install linux-image-6.12.43+deb12-
linux-image-6.12.43+deb12-amd64 linux-image-6.12.43+deb12-cloud-amd64-unsigned
linux-image-6.12.43+deb12-amd64-dbg linux-image-6.12.43+deb12-rt-amd64
linux-image-6.12.43+deb12-amd64-unsigned linux-image-6.12.43+deb12-rt-amd64-dbg
linux-image-6.12.43+deb12-cloud-amd64 linux-image-6.12.43+deb12-rt-amd64-unsigned
linux-image-6.12.43+deb12-cloud-amd64-dbg
[07:30 22] xoa:~$ apt install linux-image-6.12.43+deb12-amd64
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
linux-image-6.12.43+deb12-amd64 : PreDepends: linux-base (>= 4.12~) but 4.9 is to be installed
E: Unable to correct problems, you have held broken packages.
@olivierlambert said in Epyc VM to VM networking slow:
Kernel version could have an impact and being unrelated to the fix we provided. Get an even more recent kernel on your XOA to test (eg one from testing)
I was not able to update the kernel due to some issue in XOA installation. I commented on this at https://xcp-ng.org/forum/post/97522
But to clarify — my goal is to reach the same performance in XOA as with our other VMs. I had assumed it lacked the kernel support, and that led to the confusion. Sorry for that.
@olivierlambert said in Epyc VM to VM networking slow:
If you want to compare perf between kernels, do that with the same number of vCPUs ideally (and also the same env, ie different kernel versions with the same iperf version)
Sure. but in this context it is not that relevant as all other VMs got a boost while XOA didnt as much.
Does the Debian 6.1 kernel that XOA uses have the backported fixes mentioned in https://xcp-ng.org/blog/2025/09/01/september-2025-maintenance-update-for-xcp-ng-8-3/
Even if it has, it is clear that more resent kernels are much faster. Why not release a XOA with more recent kernels?
@bleader said in Epyc VM to VM networking slow:
@Forza By default XOA VM has 2 vcpus, how many vcpus do your ubuntu have? Althrough iperf isn't running multithreaded in your test, there is one queue on the kernel side of the VM per vcpu to process packets.
I have 8 CPUS on XOA, 4 CPUs in the Alpine VM and 6 in the Ubuntu VM
P4 uses 4 threads with iperf3.
@bleader Are you saying you do not see any performance differences between XOA and VMs with more recent kernels?
@olivierlambert said in Epyc VM to VM networking slow:
XOA's kernel should have the capability already, as it's a Debian 12 with stock kernel. Also, the bottleneck is ONLY between VMs on the same host.
OK, but then I do not understand the huge difference.
Tested some VM-VM traffic on the same host:
Ubuntu (kernel 6.8) -> Alpine (kernel 6.12): 13.7Gbit/s, P4 = 23.8 Gbit/s
XOA (kernel 6.1) -> Ubuntu (kernel 6.8) : 5.5 Gbit/s, P4 = 17.5 Gbit/s
XOA (kernel 6.1) -> Alpine (kernel 6.12): 11.9 Gbit/s, P4 = 13.6 Gbit/s
And for measure against the bare metal NFS server:
Alpine -> NFS SR (kernel 6.12): 13.7 Gbit/s, P4 = 23.4 Gbit/s
@stormi said in Epyc VM to VM networking slow:
@Forza What patch are you referring to that would relate to XOA?
It seems only recent kernels can take advantage of the improvements. From the blog post mentioned above:
The change only affects Linux guests. To make it effective, their kernel must support the feature which enables this fix.
Iperf inside XOA is much slower than other VMs (like the Ubundu 24.04 above).
When I run on the same host (pool master) against our NFS SR:
Ubuntu 24.04 (kernel 6.8):
# iperf3 -c 10.12.9.4
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 16.0 GBytes 13.7 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 16.0 GBytes 13.7 Gbits/sec receiver
# iperf3 -c 10.12.9.4 P4
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 6.26 GBytes 5.37 Gbits/sec 8861 sender
[ 5] 0.00-10.00 sec 6.25 GBytes 5.37 Gbits/sec receiver
[ 7] 0.00-10.00 sec 8.57 GBytes 7.36 Gbits/sec 8372 sender
[ 7] 0.00-10.00 sec 8.57 GBytes 7.36 Gbits/sec receiver
[ 9] 0.00-10.00 sec 9.05 GBytes 7.77 Gbits/sec 10192 sender
[ 9] 0.00-10.00 sec 9.05 GBytes 7.77 Gbits/sec receiver
[ 11] 0.00-10.00 sec 6.12 GBytes 5.25 Gbits/sec 7144 sender
[ 11] 0.00-10.00 sec 6.11 GBytes 5.25 Gbits/sec receiver
[SUM] 0.00-10.00 sec 30.0 GBytes 25.8 Gbits/sec 34569 sender
[SUM] 0.00-10.00 sec 30.0 GBytes 25.8 Gbits/sec receiver
XOA 2025.08 (kernel 6.1):
# iperf3 -c 10.12.9.4
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 6.26 GBytes 5.37 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 6.25 GBytes 5.37 Gbits/sec receiver
# iperf3 -c 10.12.9.4 -P4
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 4.71 GBytes 4.05 Gbits/sec 3987 sender
[ 5] 0.00-10.00 sec 4.71 GBytes 4.05 Gbits/sec receiver
[ 7] 0.00-10.00 sec 4.61 GBytes 3.96 Gbits/sec 3086 sender
[ 7] 0.00-10.00 sec 4.61 GBytes 3.96 Gbits/sec receiver
[ 9] 0.00-10.00 sec 6.77 GBytes 5.81 Gbits/sec 7745 sender
[ 9] 0.00-10.00 sec 6.77 GBytes 5.81 Gbits/sec receiver
[ 11] 0.00-10.00 sec 5.42 GBytes 4.65 Gbits/sec 629 sender
[ 11] 0.00-10.00 sec 5.42 GBytes 4.65 Gbits/sec receiver
[SUM] 0.00-10.00 sec 21.5 GBytes 18.5 Gbits/sec 15447 sender
[SUM] 0.00-10.00 sec 21.5 GBytes 18.5 Gbits/sec receiver
@olivierlambert Can we get an updated XOA with this patch?
@olivierlambert said in Migrating logs and history between XOA instances?:
Hi,
- VM stats aren't store on XOA but on your pools, and they should be there with the same history as the old one
- Can you tell exactly what kind of logs are you talking about?
Thanks. I had looked on the wrong VM. The stats are indeed retained.
Originally I was thinking of audit logs, but I realized they can be exported and imported. So for m point of view it is enough for now.
Other logs that are missing is for example backup logs.
@Danp said in Update XOA OS via apt update?:
@olivierlambert said in Update XOA OS via apt update?:
unintended updates
I think Olivier meant "unattended".
Haha, quite the difference 
If I understand correctly:
apt.conf.A side note. I wanted to update the kernel to see if it helps the networking speed (EPYC fix). I tried to install linux-image-6.12, but it says it isn't possible due to too old linux-base.
@olivierlambert unintended? Do you mean it is updating by itself even though it was not intended?
@Danp It is on Debian 12. I was just not aware if updates to the XOA base OS was a thing or not. It is not mentioned in the docs as far as I could look up.
@Danp said in Update XOA OS via apt update?:
@Forza This isn't needed because the XOA VM is already performing these OS updates using
unattended-upgrade.
Hm.. I have no proxy set in apt.conf, so it is not likely that is working. But If I do, will it automatically also update the base OS? And if so, could I force an update using apt update + apt upgrade?
I am updating our XOA appliance by deplying a new XOA and importing the config file. Everything seems to work alright, except that I do not have access to the logs or the VM statistics (RAM usage, etc). Is it possible to migrate these logs to the new XOA or use a central repository to store them instead of using the local XOA itself?
Is it recommended to update the XOA VM using apt update? The XOA updates via the webui (XOA->Updates page) do not contain OS level updates, only the XO server parts.
Do you mean this is why it took time or why the VHD was bigher than maximum memory allocated for the VM? The first VM I tried on was not using dynamic memory.