I've been spending some time on it for the last few nights, and my skills are improving little by little.
I think Ubuntu desktop was the closest I got to functional, there is a custom image for the Rpi3b+ that I found and it was one of the attempts that got to trying to start the XO-server service and failed. I was never able to get the xo-server to start manually either and the log files didn't give me anything I understood.
One of the versions that worked the least was CentOS 7 for RPi, node.js is not available for it in the normal repos. I have more experience with CentOS from a few years ago and more comfortable with it, may have to look into compiling all the dependencies from source to see if I can make it work.
I think an RPi 4b with 2gb or 4gb of RAM might be a better choice, but I have the 3b+.