@nikade We've already fixed some issues to start using ocaml 5 regarding the C interfaces. Handling threading in ocaml 5 is still an open problem that the ecosystem has not yet solved (there are many libraries competing now). We still need to create a credible strategy to port xapi to the new model, and don't have any timelines yet

Yea porting it is probably not a bad idea, but I guess that involves the biggest tasks as well
All tho if you port it there might be opportunities to re-do a lot of things that you have been looking to fix for ages.

Crossing my fingers that this is something that you get time to do in the near future, there is a real chance here to take a big part of the esxi market as a lot of ppl are migrating.

@john-c This involves waiting on a stable OCaml 5 version being release, then porting the whole codebase to use the new multithreading APIs, this is a very complex matter, and it will take a long time until it's realised

Is there any plans on that or is it already on some kind of roadmap?

So it doesn't seem to be an issue: XAPI will spawn multiple stunnel process if needed. XAPI itself is not multithreaded, so the bottleneck might not be stunnel after all.

@psafont Maybe worth multithreading the XAPI to help its performance, so that when going through stunnel it doesn't act as a bottleneck.

https://www.stunnel.org/NEWS.html

Using ldd on 8.2.1's stunnel binary shows dependencies on libpthread and libwrap so I'm guessing it was compiled with thread and wrapper support.

IDK and that's why I invoked @psafont

Alright!
This might actually be a big deal if its single threaded and you guys are able to bundle one that is multithreaded in the base XCP-NG installation.
Crossing my fingers over here!

@nikade said in Stunnel - Future plans to use something else?:

@john-c said in Stunnel - Future plans to use something else?:

@nikade @olivierlambert The stunnel software isn't just single threaded it is capable of multithreading already. The functionality of it depends on how stunnel is compiled, then configured for use afterwards.

The following requirements need to be met:-

• Compiled with thread support
• Create or edit your stunnel configuration file
• Ensure that the configuration file includes at least the section name and accept option.
• Running stunnel in daemon mode over inetd mode
• Optionally include TCPWrappers support when compiling or using pre compiled packages.
• Ensure libwrap library is installed (stunnel will use lwrap when present)
• Specify the allowed machines in /etc/hosts.allow in the following format:

service1: goodhost1.example.com .trusteddomain.example.com
service2: goodhost2.example.com 192.168.0.50

• Restarting stunnel (if running as a daemon already) or stopping and then starting again.

https://www.stunnel.org/howto.html
https://ipcamtalk.com/threads/stunnel-can-be-a-multi-host-incoming-router.36710/
https://stackoverflow.com/questions/54450367/stunnel-two-ubuntu-machines-traffic-encryption
https://www.stunnel.org/config_windows.html
https://www.stunnel.org/faq.html
https://superuser.com/questions/503553/how-to-properly-use-stunnel
https://www.stunnel.org/config_unix.html
https://www.stunnel.org/perf.html

Great feedback, thanks!
Do you think that this is something that could be done in the base configuration to enable better performance or is it just bandage on the problem?

BR N

It's done on a per service basis, though parts may be able to be in the base configuration. Each of those when put together enables the stunnel's multithreading, but only if the stunnel has been compiled at minimum with the thread option.

So if you are using a precompiled binary which has been packaged for your distribution, then it will need to have been compiled with the thread option before being packaged. Though it can be compiled from source with that option, though if receiving support best see if the people who are providing the support (for the distro) can provide a version with it.

@olivierlambert Do you know if it is still compiled as single threaded in XCP-NG 8.3?

@john-c said in Stunnel - Future plans to use something else?:

@nikade @olivierlambert The stunnel software isn't just single threaded it is capable of multithreading already. The functionality of it depends on how stunnel is compiled, then configured for use afterwards.

The following requirements need to be met:-

• Compiled with thread support
• Create or edit your stunnel configuration file
• Ensure that the configuration file includes at least the section name and accept option.
• Running stunnel in daemon mode over inetd mode
• Optionally include TCPWrappers support when compiling or using pre compiled packages.
• Ensure libwrap library is installed (stunnel will use lwrap when present)
• Specify the allowed machines in /etc/hosts.allow in the following format:

service1: goodhost1.example.com .trusteddomain.example.com
service2: goodhost2.example.com 192.168.0.50

• Restarting stunnel (if running as a daemon already) or stopping and then starting again.

https://www.stunnel.org/howto.html
https://ipcamtalk.com/threads/stunnel-can-be-a-multi-host-incoming-router.36710/
https://stackoverflow.com/questions/54450367/stunnel-two-ubuntu-machines-traffic-encryption
https://www.stunnel.org/config_windows.html
https://www.stunnel.org/faq.html
https://superuser.com/questions/503553/how-to-properly-use-stunnel
https://www.stunnel.org/config_unix.html
https://www.stunnel.org/perf.html

Great feedback, thanks!
Do you think that this is something that could be done in the base configuration to enable better performance or is it just bandage on the problem?

BR N

It's done on a per service basis, though parts may be able to be in the base configuration. Each of those when put together enables the stunnel's multithreading, but only if the stunnel has been compiled at minimum with the thread option.

So if you are using a precompiled binary which has been packaged for your distribution, then it will need to have been compiled with the thread option before being packaged. Though it can be compiled from source with that option, though if receiving support best see if the people who are providing the support (for the distro) can provide a version with it.

@nikade @olivierlambert The stunnel software isn't just single threaded it is capable of multithreading already. The functionality of it depends on how stunnel is compiled, then configured for use afterwards.

The following requirements need to be met:-

• Compiled with thread support
• Create or edit your stunnel configuration file
• Ensure that the configuration file includes at least the section name and accept option.
• Running stunnel in daemon mode over inetd mode
• Optionally include TCPWrappers support when compiling or using pre compiled packages.
• Ensure libwrap library is installed (stunnel will use lwrap when present)
• Specify the allowed machines in /etc/hosts.allow in the following format:

service1: goodhost1.example.com .trusteddomain.example.com
service2: goodhost2.example.com 192.168.0.50

• Restarting stunnel (if running as a daemon already) or stopping and then starting again.

https://www.stunnel.org/howto.html
https://ipcamtalk.com/threads/stunnel-can-be-a-multi-host-incoming-router.36710/
https://stackoverflow.com/questions/54450367/stunnel-two-ubuntu-machines-traffic-encryption
https://www.stunnel.org/config_windows.html
https://www.stunnel.org/faq.html
https://superuser.com/questions/503553/how-to-properly-use-stunnel
https://www.stunnel.org/config_unix.html
https://www.stunnel.org/perf.html

Great feedback, thanks!
Do you think that this is something that could be done in the base configuration to enable better performance or is it just bandage on the problem?

BR N

The following requirements need to be met:-

• Compiled with thread support
• Create or edit your stunnel configuration file
• Ensure that the configuration file includes at least the section name and accept option.
• Running stunnel in daemon mode over inetd mode
• Optionally include TCPWrappers support when compiling or using pre compiled packages.
• Ensure libwrap library is installed (stunnel will use lwrap when present)
• Specify the allowed machines in /etc/hosts.allow in the following format:

service1: goodhost1.example.com .trusteddomain.example.com
service2: goodhost2.example.com 192.168.0.50

• Restarting stunnel (if running as a daemon already) or stopping and then starting again.

https://www.stunnel.org/howto.html
https://ipcamtalk.com/threads/stunnel-can-be-a-multi-host-incoming-router.36710/
https://stackoverflow.com/questions/54450367/stunnel-two-ubuntu-machines-traffic-encryption
https://www.stunnel.org/config_windows.html
https://www.stunnel.org/faq.html
https://superuser.com/questions/503553/how-to-properly-use-stunnel
https://www.stunnel.org/config_unix.html
https://www.stunnel.org/perf.html

Hmm question for @psafont because I think I remember a discussion on how XAPI will support SSL natively in the future (but it's been a while so I could have a bad recollection of this discussion)

This would be a good thing to address - I think it would increase performance a lot since stunnel currently seems to be single-threaded and I often see it maxed out at 100% on our pools.
Atleast get it on the roadmap to make sure there is a long term plan for it, please.