how to build blktap from sources?



  • Hi all,
    I need to fiddle a bit with blktap and vhd-util, does anyone has a document to build from the sources? I have heard that @r1 has done it.

    Thanks for you help.

    Nico.



  • Hey @nraynaud

    You can rebuild it by following https://hub.docker.com/r/xenserver/xenserver-build-env/

    # docker pull xenserver/xenserver-build-env
    # wget "https://raw.githubusercontent.com/xenserver/xenserver-build-env/master/run.py"
    # ./run.py
    sh-4.2$ sudo yum-builddep blktap
    sh-4.2$ yumdownloader --source blktap
    sh-4.2$ mv blktap-3.7.0-1.0.src.rpm local-SRPMs/
    sh-4.2$ rpmbuild --rebuild local-SRPMs/blktap-3.7.0-1.0.src.rpm

    Even after this you may run into some compile time header files issue e.g.
    td-req.h:37:24: fatal error: xen/gntdev.h: No such file or directory #include <xen/gntdev.h>

    Although above is the official method - it may not succeed.

    I used a DDK VM and used whatever blktap rpm/git repo to rebuild. That DDK VM had a locally mounted source.iso to rebuild everything possible. This is a lot lengthy process than the docker one but I managed to make this work. YMMV.

    Feel free to report your obstacles and I will be happy to help.



  • thanks, I've been inching along with external help, I just managed to compile the package a few minutes ago.
    I will try to report here with an exact script to compile the RPM from nothing.



  • The solution was to install kernel-headers in the container before rpm rebuild.

    sh-4.2$ sudo yum install centos-release-xen.x86_64
    sh-4.2$ sudo yum install kernel-headers
    and then
    sh-4.2$ rpmbuild --rebuild local-SRPMs/blktap-3.7.0-1.0.src.rpm

    This should give you the blktap package.

    Cheers.



  • Who can mark this as solved?



  • It's now done 🙂



  • @r1 Why is centos-release-xen.x86_64 needed?

    Also, I found out that in order for blktap to build, I had to install kernel-headers from a rebuilt kernel from the source RPM from XS source ISO. It won't work with stock centos kernel, at least not when starting the container with "docker run" rather than "./run.py" (the latter adds a useful repository containing build dependencies for most xenserver-related packages, but its contents is evolving as development continues, so in order to build packages for XCP-ng 7.5, I prefer to avoid relying on it. And of course the similar build repository for the branch that corresponds to XS 7.5 is internal to Citrix only).



  • @stormi I had discussion on GitHub with Citrix dev who recommended to use either latest kernel headers or one can use them from a rebuilt kernel. Rebuilt kernel headers is the right way to do this.



  • Can you provide a bit more info, how to do it?

    I managed to build the package, but it was not the right version and with the wrong headers. Do you have something like a commandlist to give me some hints?

    I'm playing around with ZFS and would build my own blktap-package to modify and test some things.



  • Yes, it's quite complicated.

    You need xenserver-build-env, and a repo of someone else's already compiled RPMs.

    I added the repo to my Dockerfile:
    COPY xcp-ng.repo /etc/yum.repos.d/xcp-ng-base.repo

    Then I created a launch script:

    #!/usr/bin/env bash
    
    (cd ../blktap && git diff 41bdd82242104ecea3f654f12704fe8122dfdf41 ) > nr-remove-odirect.patch
    docker build -t xenserver/xenserver-build-env .
    python run.py -v `pwd`:/builddir -v `pwd`/repo-7.4:/repo-7.4 -s /Volumes/7.4.0\ Source/SPackages/blktap-3.5.0-1.12.src.rpm /builddir/build-blktap-inside-docker.sh
    

    build-blktap-inside-docker.sh is here:

    #!/usr/bin/env bash
    
    set -ex
    
    sudo yum -y remove kernel-headers
    sudo rpm -ivh /repo-7.4/x86_64/kernel-headers-4.4.52-3.2.2.x86_64.rpm
    rpm -ivh local-SRPMs/blktap-3.5.0-1.12.src.rpm
    sudo yum-builddep -y blktap
    
    cp /builddir/nr-remove-odirect.patch rpmbuild/SOURCES/
    cp /builddir/blktap.spec rpmbuild/SPECS/
    rpmbuild -ba rpmbuild/SPECS/blktap.spec
    cp rpmbuild/RPMS/x86_64/blktap-3.5.0-1.12test.x86_64.rpm /builddir
    

    The specfile just has a patch added and its version changed:

    Patch54: nr-remove-odirect.patch
    

    good luck.



  • hui ... will try that ...



  • @nraynaud said in how to build blktap from sources?:

    repo-7.4

    whats inside this directory? content of the XenServer 7.4 source iso?





  • ok, got it working! my own blktap is running 🙂 Thanks for your help!



  • @borzel sorry, I missed your question, glad you got it working.

    what changes are you making?



  • First just playing around to get knowledge of this, maybe I find some usefull things. Don't know exactly, but I want to understand a bit more what I'm using here.

    Can sometimes be handy to know such things. Everytime I learned something I was using it later at work or privat 🙂



  • Good idea 🙂 We are glad to have more potential contributors on board. More the XCP-ng knowledge is spread, better the community!



  • @borzel Congratulations! And I can relate to the feeling 🙂



  • There's now a new and reproducible way to build blktap in a clean environment:

    https://github.com/xcp-ng/xcp-ng-build-env

    I will not copy the readme here, but basically, you can do something like that:

    Preparation

    # create the build image, target current dev branch of XCP-ng (which is the future 7.5)
    # There's no support for 7.4 because of different repository structure and missing devel packages
    # You need to do it just once, though doing it from time to time won't hurt.
    ./build.sh dev
    
    # create an useful alias for later. You should probably add it to you .bashrc so that it stays
    # The --rm part is not mandatory but I find it useful to keep docker's disk usage low.
    alias xcp-build="$(pwd)/run.py --rm"
    

    Build

    Here I'll show you how to build from our new git repositories. Not all packages are present there, but blktap is and this is a good occasion to make you become accustomed to using them since that's where we'll commit patches and also experimental stuff.

    # from the directory of your choice...
    git clone https://github.com/xcp-ng-rpms/blktap.git # or a forked version if you intend to send pull requests
    cd blktap
    # ... here make changes... or not...
    # use our shiny new alias to build it
    xcp-build --build-local . # or simply 'xcp-build -l .'
    

    The resulting built RPMs will be in the SRPMS and RPMS subdirectories of your git working directory.

    Just tested with blktap. If it fails at building another package, open a separate thread and I'll try to fix it.

    Bonus: build with experimental changes applied

    Our blktap package can be built with an experimental patch for better ZFS support. Build it this way:

    xcp-build --build-local . --define 'xcp_ng_section extras'
    


  • If someone fails to build, maybe the git-lfs in not properly initialised. To fetch the lfs-files after git clone you can do a git lfs fetch.
    weird ... after the second time installing git-lfs and again cloning the repo, it worked... 😮