CEPH FS Storage Driver



  • Can you show your mount command by doing some example mount? e.g. #mount.ceph add1:remotepath localpath /mnt? Does this work as it is?

    The advanced options are necessary for now to discriminate between NFS and Ceph. But I think once I know your working command, I will be able to generalize it.

    BTW - I hope you installed the ceph.ko as mentioned in earlier steps.



  • @r1 I'm in planning phase right now since these boxes are actually production and I'll have to figure how to do this without it being a possible danger to my production enviro. The box I have that needs it the worst I still haven't upgraded from xenserver 6.5. I need to setup a fully dev box for this. I've got plans for upgrading my infrastructure for the ceph and virtual public network that this would help tremendously.



  • Sadly XS 6.5 uses 3.10 kernel which does not support Ceph outright and you would most likely to update this host to a recent XCP version.



  • @r1 Yeah I just haven't had time yet since it has one of our heaviest used virtuals on it and not good to upset the masses lol.


  • XCP-ng Center Team

    @scboley said in CEPH FS Storage Driver:

    and not good to upset the masses

    this is the reason I worked at night the last two weeks at work 🌃



  • @r1 sudo mount -t ceph sanadmin.nams.net:6789: / /mnt/nfsmigrate



  • @scboley nice. It should work straight as mentioned in 2nd post of this thread.



  • Ok,

    i start my adventure but I get soon an issue.

    This command install the nautilus release Repo:

    yum install centos-release-ceph-nautilus --enablerepo=extras
    

    and worked fine.

    This one should installed ceph-common but cannot find many dependencies

    yum install ceph-common
    Loaded plugins: fastestmirror
    centos-ceph-nautilus                                                                                                                                | 2.9 kB  00:00:00
    centos-ceph-nautilus/7/x86_64/primary_db                                                                                                            |  62 kB  00:00:00
    Loading mirror speeds from cached hostfile
     * centos-ceph-nautilus: mirrors.prometeus.net
    Resolving Dependencies
    --> Running transaction check
    ---> Package ceph-common.x86_64 2:14.2.0-1.el7 will be installed
    --> Processing Dependency: python-rgw = 2:14.2.0-1.el7 for package: 2:ceph-common-14.2.0-1.el7.x86_64
    
    ... [CUT] ...
    
    Error: Package: 2:librgw2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1()(64bit)
    Error: Package: leveldb-1.12.0-5.el7.1.x86_64 (centos-ceph-nautilus)
               Requires: libsnappy.so.1()(64bit)
    Error: Package: 2:libcephfs2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1()(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: liblz4.so.1()(64bit)
    Error: Package: 2:libradosstriper1-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1()(64bit)
    Error: Package: 2:librbd1-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1()(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: python-requests
    Error: Package: 2:librgw2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1()(64bit)
    Error: Package: 2:librados2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1()(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1()(64bit)
    Error: Package: 2:libcephfs2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1()(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libfuse.so.2()(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libsnappy.so.1()(64bit)
    Error: Package: 2:librbd1-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1()(64bit)
    Error: Package: 2:librados2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1(RDMACM_1.0)(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librabbitmq.so.4()(64bit)
    Error: Package: 2:libradosstriper1-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1()(64bit)
    Error: Package: 2:librgw2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librabbitmq.so.4()(64bit)
    Error: Package: 2:librados2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1()(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libtcmalloc.so.4()(64bit)
    Error: Package: 2:librados2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1(IBVERBS_1.0)(64bit)
    Error: Package: 2:librados2-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: libibverbs.so.1(IBVERBS_1.1)(64bit)
    Error: Package: 2:ceph-common-14.2.0-1.el7.x86_64 (centos-ceph-nautilus)
               Requires: librdmacm.so.1()(64bit)
     You could try using --skip-broken to work around the problem
     You could try running: rpm -Va --nofiles --nodigest
    
    

    Of course I'm using a higher version of Ceph than your post.
    Any hint anyway?



  • Ok, solved.
    Of course Ceph need Epel and Base repo enabled.

    This worked:

    yum install epel-release -y --enablerepo=extras
    yum install ceph-common --enablerepo='centos-ceph-nautilus' --enablerepo='epel' --enablerepo='base'
    


  • @r1 said in CEPH FS Storage Driver:

    patch -p0 < ceph.patch

    This went not so smooth, reporting "unexpected end":

    patch -p0 < ceph.patch
    patching file /opt/xensource/sm/nfs.py
    patching file /opt/xensource/sm/NFSSR.py
    patch unexpectedly ends in middle of line
    Hunk #3 succeeded at 197 with fuzz 1.
    


  • Hi @maxcuttins - what's your XCP-NG version and can you share checksum for your NFSSR.py file?


Log in to reply