@olivierlambert
Thank you again for the answer!
I was not using the right search criteria to find this part myself
Changed the 'config.toml' file after searching my system (not an XOA but XO CE, so it was in another location then mentioned in the documentation)
Restarted and just as expected the GUI was only available on the specified IP