Troubleshooting "TCP: out of memory" - Possible memory leak?
-
Did a much closer look and can indeed confirm that the network capabilities go up from kb/s -> MB/s immediately from xo_server restarts.
-
On the tuning suggestion, my guess, untested and purely from reading your paste, is that if something is genuinely holding sockets then raising
tcp_membuys headroom rather than stopping the accumulation, and your post#4 about throughput coming back on anxo_serverrestart points the same way.
It might be worth a mention to @Team-XO-Backend, since they would know whether
xo-serverkeeps sockets per pool. -
On the tuning suggestion, my guess, untested and purely from reading your paste, is that if something is genuinely holding sockets then raising
tcp_membuys headroom rather than stopping the accumulation, and your post#4 about throughput coming back on anxo_serverrestart points the same way.It might be worth a mention to @Team-XO-Backend, since they would know whether
xo-serverkeeps sockets per pool.This is why I'm not increasing the numbers quite yet. Since this would only mask a potential issue. If an underlying cause is responsible, then this would be of interest.
Thanks for chiming in!
-
Looking at it today again. Doing a
sudo ss -tpand noticing that there are alot ofCLOSE-WAIT. Maybe this is a red herring, or maybe not. I thought I would share the findings regardless.The output is also to long for me to be able to paste it in text. So I've attached it as a txt-file.
close-wait.txt -
@jr-m4 you can export the heap memory of the nodeJS process by doing
kill -SIGUSER2 <xoserverpid>onte that this will increase a lot the memory consumed by the xo process even when its done exporting the memory . This will help us know what xo is doing at the moment
-
-
@jr-m4 you can export the heap memory of the nodeJS process by doing
kill -SIGUSER2 <xoserverpid>onte that this will increase a lot the memory consumed by the xo process even when its done exporting the memory . This will help us know what xo is doing at the moment
Do you have somewhere I can upload the heapsnapshot? (148MB)
Ping @poddingue as well, for visibility
-
I now have two heap snapshots. Could you provide somewhere to upload them, please? And I will do so as soon as I can.
-
I have now uploaded two heap-snapshots for you to look at.
If there is anything else, big or small, that I can help out with. Just let me knowThanks!
-
Forwarded to the right team, thanks!
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login