Hi Everyone,

As you know, XCP-NG uses a highly modified kernel 4.4. A lot of device drivers are already present in this kernel tree, however there are edge cases where one wants to add a new driver.

We have added a simple repo which you can fork, add your driver source code and let XCP-NG build system do the rest of work for you

The output is located in RPMS folder, which you can ship to your XCP-NG host and install.

Please have a look https://github.com/xcp-ng-rpms/kernel-driver-template and share your feedback on being able to build and test your first driver.

@olivierlambert said in XenServer 8.0 - Major update due Q1 2019:

Our ambition is to become the first turnkey and Open Source virtualization platform.

And the community involved here is progressing to achieve this collectively.

@stormi happy to have you. Hope to see some great action!

So taking inspiration and doing small modifications to existing NFS driver, patch allows you to use both NFS and Ceph FS same time.

Note : The patch is applied against XCP-NG 7.6.0

# cd /
# wget "https://gist.githubusercontent.com/rushikeshjadhav/af53bb5747365875f0ab21bd3a64c6fe/raw/59ef7a4b54574e4163da1ac39acd640554bd0d24/ceph.patch"
# patch -p0 < ceph.patch

Apart from the patch, ceph.ko & ceph-common would be needed.

To install ceph-common on XCP-NG
# yum install centos-release-ceph-luminous --enablerepo=extras
# yum install ceph-common

To install and load ceph.ko
# wget -O ceph-4.4.52-4.0.12.x86_64.rpm "https://github.com/rushikeshjadhav/ceph-4.4.52/blob/master/RPMS/x86_64/ceph-4.4.52-4.0.12.x86_64.rpm?raw=true"
# yum install ceph-4.4.52-4.0.12.x86_64.rpm
# modprobe -v ceph (Optional)

User experience:

In Share Name, you can put multiple addresses from your CEPH mon.
In Advanced Options, you need to add your ceph authx user details, such as name and its secret file location. These will be used by mount.ceph.

Note: Keep secret file in /etc/ceph/ with permissions 600.

Current SR detection patch replies on presence of word "ceph" in advance options to discriminate between NFS and CEPH FS.

Log Location : /var/log/SMlog

Edit: Created RPM installer for ceph.ko

As a side experiment - I could extract and test CEPH FS kernel module for XCP-NG 7.4+ and mount a multi-mon ceph cluster (luminous).

Once ceph.ko module is in place, XCP-NG can mount the ceph fs mount point similar to NFS.

e.g. In NFSSR we use #mount.nfs4 addr:remotepath localpath

while for CEPHSR we can use #mount.ceph addr1,addr2,addr3,addr4:remotepath localpath

I'm currently looking at NFSSR.py to create CEPHFSSR.py - and would share once ready, meanwhile if anyone wants to help me in this by testing ceph.ko, developing CEPHFSSR.py, kindly ping here. Like EXT4 and XFS, this too will have some cleanup issues..

If this works as expected, I'd request @olivierlambert and @borzel to look at possibilities of integrating CEPH FS option in XO/XC near NFS SR.

Note: This is completely different than RBDSR which is worked by @rposudnevskiy and uses rbd image protocol of ceph.

Bonus : If someone knows how nfs-ganesha fits in this - buzzz... e.g. Each CEPH FS node can have a NFS server while each XCP-NG host mounts all of them using NFS 4.1 (pNFS). Bypassing need of ceph.ko. We can also rebuild NFS module as CONFIG_NFSD_PNFS is not set by default.

Caveats : CEPH FS caching is not fully explored.

State: Ready. See post below for instructions on how to use.

@jmccoy555 I did not continue this. Will definitely rejuvenate it once 8.1 is released. Hopefully next week.

@olivierlambert said in Windows 10 VM reboots when starting software:

I don't know if @r1 can build it for us, but we'll have to wait next week for @stormi to put the package in testing anyway

The patch needed a rework for fitment. I'd create a PR for modified patch and then it can go in testing repo.

Yes - its in xcp-ng-testing repo which is not enabled by default.

Link https://github.com/xcp-ng/xcp/wiki/Alternate-kernel is now updated with installation instructions.

@dave-opc To know more on next incident, you may add noreboot options to multiboot2 /boot/xen.gz in /etc/grub.cfg file against default kernel boot entry.

@dave-opc whats the XCP-ng version?

Did you see -

https://github.com/xcp-ng/xcp/wiki/Troubleshooting

https://github.com/xcp-ng/xcp/wiki/Logfiles

Hi @achim71 @stormi

Patching only for 1 issue would leave other issues - just waiting to be found.

Linux kernel gets weekly updates for fixes to stable and longterm. New features get added in new versions. I think at XCP we can follow longterm branch of 4.19 and keep pushing updates of kernels on regular basis.

I have at tracked till 4.19.48 at link and at 2 instances I found that using the next patch on top of existing patches needs certain modifications. ( e.g. 37-pre and 47-pre )

Following Linux longterm in XCP also means going back to CH updates will not be possible. But I assume, CH will anyways take updates from Linux longterm.

@tjkreidl Hearty welcome to the community. We look forward for your insights and expert advises. Cheers

Ah! You will have to do a #interface-rename operation to rename your side-373-eth0 to eth1, then XAPI will recognize it. Can you try it, I don't have exact command around but you can check with its help or code.

Congratulations #TeamXCPNG

Will be interesting to see what all major changes they bring.

@jmccoy555 glad to know. I don't have much knowledge on File Layouts but that looks good.

NFS edits won't be merged as that was just for POC. Working on a dedicated CephFS SR driver which hopefully won't be impacted due to sm or other upgrades. Keep watching this space.

@jmccoy555 said in CEPH FS Storage Driver:

Unfortunately at the moment the patch breaks reconnecting the NFS SR,

It did not intend to.. I'll recheck on this.

@Marc-pezin Thanks. I'd suggest to also add # xe-toolstack-restart so that Error code: MISSING_URI does not show up

@Marc-pezin said in SMAPIv3 - Feedback & Bug reports:

yum install xcp-ng-xapi-storage

This needs to be # yum install xcp-ng-xapi-storage --enablerepo=xcp-ng-testing

I tried following

[13:33 xcp-ng-8-kernel ~]# yum install xcp-ng-xapi-storage --enablerepo=xcp-ng-testing
...
[13:34 xcp-ng-8-kernel ~]# systemctl start qemuback.service
[13:34 xcp-ng-8-kernel ~]# systemctl status qemuback.service
● qemuback.service - qemuback daemon
   Loaded: loaded (/usr/lib/systemd/system/qemuback.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/qemuback.service.d
           └─slice.conf
   Active: active (running) since Sat 2019-10-05 13:34:59 IST; 5s ago
 Main PID: 10221 (qemuback.py)
   CGroup: /control.slice/qemuback.service
           ├─10221 /usr/bin/python /usr/bin/qemuback.py
           └─10222 /usr/bin/xenstore-watch /local/domain/0/backend

[13:35 xcp-ng-8-kernel ~]# xe sr-create type=filebased name-label=sr-test2 device-config:file-uri=/root/sr-folder
c6e9c83c-78ee-7727-dec3-1c7474aef533
[13:36 xcp-ng-8-kernel ~]# ls sr-folder/
1  db_backups  lock_db  lock_db_backup  lock_gl  meta.json  meta.lock  sqlite3-metadata.db
[13:37 xcp-ng-8-kernel ~]# xe vbd-create vdi-uuid=cb6cf26c-b079-44e2-9466-3d61fc7ac553 vm-uuid=a00c837e-20ab-4b3b-b125-2a73df1070d2 device=5
6877ea3b-d3c7-e708-5b34-c5e61e5dd1eb
[13:37 xcp-ng-8-kernel ~]# xe vbd-plug uuid=6877ea3b-d3c7-e708-5b34-c5e61e5dd1eb 
Error code: MISSING_URI
Error parameters: Please include a URI in the device-config


FYI
[13:38 xcp-ng-8-kernel ~]# xe vdi-list uuid=cb6cf26c-b079-44e2-9466-3d61fc7ac553 
uuid ( RO)                : cb6cf26c-b079-44e2-9466-3d61fc7ac553
          name-label ( RW): New virtual disk (1)
    name-description ( RW): 
             sr-uuid ( RO): c6e9c83c-78ee-7727-dec3-1c7474aef533
        virtual-size ( RO): 1073741824
            sharable ( RO): false
           read-only ( RO): false

What can be done for Error code: MISSING_URI?

Edit: I did not read the restart instruction.

After restarting tool stack it worked and disk attached as /dev/nbd0!

[13:55 xcp-ng-8-kernel ~]# xe-toolstack-restart 
Executing xe-toolstack-restart
done.
[13:56 xcp-ng-8-kernel ~]# xe vbd-plug uuid=6877ea3b-d3c7-e708-5b34-c5e61e5dd1eb
 

@alphastarr128 glad I could help.

@olivierlambert @stormi Looking forward to it!

Cheers.