Very scary host reboot issue
-
@olivierlambert said in Very scary host reboot issue:
FreeBSD PV driver inside OPNsense or Pfsense.
Who is maintaining the FreeBSD PV drivers?
-
Maybe it's time to ask OPNsense devs
-
@olivierlambert Last time I asked about a Xen driver issue, they deferred to the FreeBSD maintainers.
-
Maybe it's time to ask FreeBSD devs, then
-
@olivierlambert I'm thinking of a quick workaround. What if I use pci pass-through for the LAN and WAN interfaces and I physically connect the LAN port to another non PCIe pass-through port of the server and I use that port toninterface with my other VMs via OVS? Does it make any sense? Does it seem viable to mitigate this issue?
-
@darabontors Yes, if you pass the PCIe LAN/WAN hardware to the VM then it will bypass the FreeBSD Xen network drivers and the Dom0 Xen OVS drivers.
FreeBSD will use its own hardware drivers for the network interfaces.
You won't be able to use the interfaces for any shared VM. You won't be able to hot migrate the VM to another host.
-
@Andrew That makes sense. I think I'll do just this. In the meantime I'll try to replicate the phenomenon on test hardware. I really need a permanent fix for this..
-
We'll be all happy when we'll find that bug
-
@darabontors some additional tests that I could think of:
- Minimum WG MTU on client-side (
MTU=1280
); - OPNSense with emulated
e1000
interfaces (bypass the PV driver but not OVS). It'll keep the VM 'agile' (hot-migrate) but with a big cost in performance; - The last OPNSense version
23.7.5
.
As for the last version, found this important info posted by the devs about a change in the MTU code [1]:
Today introduces a change in MTU handling for parent interfaces mostly
noticed by PPPoE use where the respective MTU values need to fit the
parent plus the additional header of the VLAN or PPPoE. Should the
MTU already be misconfigured to a smaller value it will be used as
configured so check your configuration and clear the MTU value if you
want the system to decide about the effective parent MTU size.
(...)Hope it helps.
- Minimum WG MTU on client-side (
-
- Absolutely no idea how to do this in Windows. I looked for any MTU setting but couldn't find any.
- This is not a viable workaround for me, maybe it would be useful to pin the issue to the xen PV driver, maybe I'll do some more testing on spare hardware.
- I read this, but I don't know how to test it. I didn't have any manual MTUs set so I don't know what values were before the update.
What most definitely fixed the issue for me was using PCIe passthrough for the WAN interface. I used a 10 GbE NIC. It uses the ix driver (ix0) so IDK if this is related. Somehow PPPoE + WG + Windows Client on the virtual interface (Xen PV driver) in OPNsense produces this issue.
At the moment I am happy with this mitigation.I'm a little spread thin with free time at the moment. Anyone care to test this further?