[solved] NFS on XO failing, on Center has no issues

olivierlambert

I have no idea how you installed XO (from the sources right?), and it looks like permissions issues, not a problem directly related to XO itself

mavoff

This XO has been built from sources on a debian VM according the documentation it worked flawlessly. (you know I spent several hours trying to build on FreeBSD, didn't work, debian worked fine).

This Remote / NFS share is configured alike the others. Inclusive the same user and permissions. I can mount the share on XCP-ng host without issues as well as using the XCP-ng Center. If it works everywhere else except on XO, it isn't an XO problem? How come?

olivierlambert

As I already said, that's because a SR is NOT a remote, it's not the same thing. It's not used the same way, it's just 2 different things.

So that's why it's not the same behavior. You can have a permission issue on the same NFS share used as a remote, that you don't have if you use it as a SR. Not the same thing.

mavoff

So exactly what do I have to consider to add the remote to XO? Is there a specific user I should configure access to the NFS share?

I don't find these processes minimally straightforward. Not looking to put 777 permissions on the share.

olivierlambert

You need to check that your current XO user for xo-server has the NFS permissions to write on the targeted folder, even for . files.

Check your NFS permissions.

mavoff

hmm... well, so xo-server is running under root as it was the option that would allow to use the standard ports.

Does this mean I have to assign access to user root to this share?

btw I have another question... is there a way of getting plugins to the XO from source? thank you

olivierlambert

This is NFS permission question, here is a good read about it: http://www.troubleshooters.com/linux/nfs.htm

mavoff

I am going to read that now.

Anyway, please allow me to share this thought:

I tried with two users: the root user and the user I use to login to XO.
None worked both returned the same error.

Many times I have expressed some unease with XO, I am requested examples and to share.
Well, this is one of those situations where I'm definitely hating XO and craving for Center.
I never had this sort of issues with Center. You add storage from various sources and it works. I am wasting hours to "fix" something that isn't broken (I can easily mount this share on CLI:

root@xo:~# mount -t nfs -o rw 10.49.10.1:/export/backups /mnt
root@xo:~# mount -l | grep backups
10.49.10.1:/export/backups on /mnt type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.49.10.2,local_lock=none,addr=10.49.10.1)

EDIT: wrong NFS version above.

root@xo:~# umount /mnt
root@xo:~# mount -t nfs -o rw -o vers=3 10.49.10.1:/export/backups /mnt
root@xo:~# mount -l | grep backups
10.49.10.1:/export/backups on /mnt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.49.10.1,mountvers=3,mountport=20048,mountproto=udp,local_lock=none,addr=10.49.10.1)

So I'll look to the link you provided, and try to figure out why xo-server can't write to a folder that itself its supposed to create. This should be handled by the orchestra software itself... imo... thanks.

olivierlambert

Okay so I'm patient and explaining this again. You are comparing 2 different things. You can't make backups with XCP-ng Center, so there's no remote concept or whatever in there. If you crave for it, use it if you like, but you won't have any backup possible. Different things.

If you create an SR with Xen Orchestra or XCP-ng Center, it's exactly the same XAPI call, you won't have any issue. So if you compare apples to apples, it works the same way

Here, you are working with something else: being able to mount NFS share from XO (something you don't do at all with XCP-ng Center). If you have high expectation on something that you can have working out of the box, you have 2 choices:

Learning how thing are working with the version from the sources. In your case, it means understand how NFS permissions are working. I have to emphasis on the fact XO is using mount command, so it's relatively standard. However, if you are disappointed because it doesn't work out-of-the-box, be advised we can't do the NFS server permission configuration for you, because it's not Xen Orchestra related, but NFS related on your server. XO can't configure your NFS server, this is something beyond our scope. It's like you are complaining about the road quality to your car dealer.
If you don't want to spend time to figure it out (and I can fully understand that), we also offer pro support and configuration with remote assistance.

mavoff

@olivierlambert first of all please let me appreciate your time and effort of trying to explain all so we understand it correctly.

I got the first part. XO is different from Center and the Remote concept doesn't apply there.

Following point 1: that's where I'm at, trying to understand how to work with XO from sources... However, notice this: my issue isn't with mount. My issue is with XO not being able to read on a location itself created, after trying with all possible users that would fit for purpose:

EACCES: permission denied, open '/run/xo-server/mounts/cb8db284-3d88-42fc-9f43-181979914b20/.keeper_i109kg76ya'

So my question is, the folder /run/xo-server/mounts/etc should be read by who? The issue isn't the mount command itself it seems to me, but reading permissions on a folder created by XO/mount.

The xo-server config has useSudo=true - tried with and without - same exact error. I have configured the NFS share for the user root and for the admin user of XO. none work.
Sorry if I'm being a bit hard on getting this, I configure the users correctly for NFS for a bunch of different stuff and I don't come across these errors. I would like to learn and understand how NFS permissions are working HERE.
I'm sure if I'd reduce the security standards this would work. And I am also sure many of those adept of 777 permissions also would have marked this solved. But that isn't "understanding how NFS permissions work (here)". I truly would love to understand what user is that that would have permission, if root doesn't work, the admin user of XO doesn't work, useSudo=true or false is the same.

Biggen

Have you actually tried to mount the NFS share from another machine and see if you can read/write to it? This sounds like an NFS permissions issue.

With NFS, you don't usually have a username/password. You can restrict access, if you wish, via IP or other measures. I have a Synology NAS unit configured as a NFS Remote for XO to send backups to. I don't specify a user/password for the share. I just feed XO the path to the share and let it rip.

mavoff

I'm not expecting any user/passwd, NFS works with User ID mapping ...

From redhat knowledge base:

nfsd bases its access control to files on the server machine on the uid and gid provided in each NFS RPC request. The normal behavior a user would expect is that she can access her files on the server just as she would on a normal file system. This requires that the same uids and gids are used on the client and the server machine. This is not always true, nor is it always desirable.

When a uid or gid on the nfs client does not exist on the nfs server, nfsd by default "squashes" the non-existent uid/gid. This means that for uids and gids that do not exist on the nfs server, exportfs chooses a uid and gid of 65534 for squashed access, which corresponds to user and group nfsnobody.

In summary, client uids/gids are used if they exist on the server. If they do not exist, they get squashed to nfsnobody. This is the default behavior.

mavoff

fixed. had a problem on export on the nfs server indeed.

olivierlambert

What was the exact issue? A typo or a bad user mapping?

mavoff

the export had the wrong anonuid and anongid bc I was taking the uid and gid from an outdated document. I felt really stupid........

olivierlambert

No problem, it happens.