XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    [SOLVED] Just FYI: current update seams to break NUT dependancies

    Scheduled Pinned Locked Moved Solved XCP-ng
    40 Posts 11 Posters 9.6k Views 11 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • rzrR
      rzr Vates 🪐 XCP-ng Team @FritzGerald
      last edited by rzr

      @FritzGerald said:

      @rzr: Sorry, first and foremost I totally forgot to say thank you very much. I really appreciate you work and effort for this "non-core" issue.

      You're welcome !

      ⚠ Keep in mind using/enabling ci repo can be dangerous but for this nut update the risk is low.

      Is there maybe anything particular you are interested in for me to test?

      Nothing specific, the regular use case but feel free to share to community your experience with your hardware:

      https://networkupstools.org/ddl/APC/Smart-UPS_750.html

      Then others can compare with other devices supported by N.U.T. :

      https://networkupstools.org/ddl/

      This makes me think, I have scavenged an APC BE400 but it looks like an offline device, unsure I will make anything useful soon.

      F 1 Reply Last reply
      Reply Quote 0
      • F
        FritzGerald
        last edited by FritzGerald

        Good point.

        Quick install procedure for anyone interested. I use a USB connected APC Smart-UPS 750 (SMT750RMI2U) on a completely patched test "XCP-ng 8.3" server.

        To install packages run:

        yum install nut
        

        Most important, check that your USV device is properly connected and found using

        lsusb
        

        it should show something like:

        ...
        Bus 002 Device 002: ID 051d:0003 American Power Conversion UPS
        ...
        

        Grab details then by:

        lsusb -s 002:002 -v
        

        which should give you something like

        [12:53 xcp-test ~]# lsusb -s 002:002 -v
        
        Bus 002 Device 002: ID 051d:0003 American Power Conversion UPS
        Device Descriptor:
          bLength                18
          bDescriptorType         1
          bcdUSB               2.00
          bDeviceClass            0 (Defined at Interface level)
          bDeviceSubClass         0
          bDeviceProtocol         0
          bMaxPacketSize0        64
          idVendor           0x051d American Power Conversion
          idProduct          0x0003 UPS
          bcdDevice            1.06
          iManufacturer           1 American Power Conversion
          iProduct                2 Smart-UPS 750 FW:UPS 15.1 / ID=18
          iSerial                 3 SERIALXXXXXXXX
          bNumConfigurations      1
        ...
          Remote Wakeup Enabled
        

        Based upon this data you can add this information to your ups.conf

        nano /etc/ups/ups.conf
        
        [apc]
                driver = "usbhid-ups"
                port = "auto"
                vendorid = "051D"
                productid = "0003"
                product = "Smart-UPS 750 FW:UPS 15.1 / ID=18"
                serial = "SERIALXXXXXXXX"
                vendor = "American Power Conversion"
                bus = "002"
        

        Now add users in nano /etc/ups/upsd.users. "upsmon" is for slave connected devices.

        [admin]
              password = PWXXXXXX
              actions = SET FSD
              instcmds = ALL
              upsmon master
        
        [upsmon]
              password = SLAVEPW
              upsmon slave
        

        Now add/enable via nano /etc/ups/upsd.conf the required lines. If you want other servers to listen, you need to listen to a non-localhost IP of course.

        LISTEN 127.0.0.1 3493
        

        Now enable in nano /etc/ups/upsmon.conf the relevant line:

        MONITOR apc@localhost 1 admin PWXXXXXX master
        

        Now restart services:

        upsdrvctl start
        systemctl start nut-server.service
        systemctl start nut-monitor.service
        

        and you should be able to see the USV data by calling:

        upsc apc@localhost
        

        It should show:

        battery.charge: 100
        battery.charge.low: 10
        battery.charge.warning: 50
        battery.runtime: 3900
        battery.runtime.low: 120
        battery.type: PbAc
        battery.voltage: 27.3
        battery.voltage.nominal: 24.0
        device.mfr: American Power Conversion
        device.model: Smart-UPS 750
        device.serial: SERIALXXXXX
        device.type: ups
        driver.name: usbhid-ups
        driver.parameter.bus: 002
        driver.parameter.pollfreq: 30
        driver.parameter.pollinterval: 2
        driver.parameter.port: auto
        driver.parameter.product: Smart-UPS 750 FW:UPS 15.1 / ID=18
        driver.parameter.productid: 0003
        driver.parameter.serial: SERIALXXXXX
        driver.parameter.synchronous: auto
        driver.parameter.vendor: American Power Conversion
        driver.parameter.vendorid: 051D
        driver.version: 2.8.0
        driver.version.data: APC HID 0.98
        driver.version.internal: 0.47
        driver.version.usb: libusb-0.1 (or compat)
        ups.beeper.status: enabled
        ups.delay.shutdown: 20
        ups.firmware: UPS 15.1 / ID=18
        ups.mfr: American Power Conversion
        ups.mfr.date: 2015/08/14
        ups.model: Smart-UPS 750
        ups.productid: 0003
        ups.serial: SERIALXXXXX
        ups.status: OL
        ups.timer.reboot: -1
        ups.timer.shutdown: -1
        ups.vendorid: 051d
        

        HINT#1: I experienced that the bus id of the usb device sometimes changes after reboot. Watch out if you restart the server and adapt your ups.conf accordingly!!!

        HINT#2: You need to open firewall port 3493 if you want to access it from a monitor!

        HINT#3: To automatically run NUT as a service follow these steps https://xcp-ng.org/forum/post/108926

        K 1 Reply Last reply
        Reply Quote 1
        • F
          FritzGerald @rzr
          last edited by

          @rzr Just for curiosity.

          As far as I understand you assess the risk as low for the nut package - and only for that - to use ci-repo. I am fine with that for my personal servers. In terms of staying as close to the stable version in order to avoid potential future upgrade issues, is there a chance to say something about:

          • Would you consider to integrate this into "stable" branch?
          • What would be the next steps on our side to support this?
          • How long would does that roughly take?

          Thank again!

          rzrR 1 Reply Last reply
          Reply Quote 0
          • rzrR
            rzr Vates 🪐 XCP-ng Team @FritzGerald
            last edited by

            @FritzGerald said:

            @rzr Just for curiosity.

            As far as I understand you assess the risk as low for the nut package - and only for that - to use ci-repo. I am fine with that for my personal servers.

            • Would you consider to integrate this into "stable" branch?

            yes it's the plan, we usually test packages in ci repos for a couple of weeks and once we're happy of the packages set it goes to testing repo, then users can officially test it (and based on feedback (or absence of feedback) then everything lands in update repo after a week or more (it depends on risk or hurry)..

            • What would be the next steps on our side to support this?

            I think nothing should be done your side you did your job before the testing window, early feedback is always appreciated (as long people know what they are doing 😉 )

            But If I were you, just to be safe i would keep trace of packages i have installed from ci or testing repo, in case anything is not going as expected, you can still reinstall those packages from update channel.

            • How long would does that roughly take?

            It depends usually a couple of weeks, but this next batch might be a bit longer because there is a long waited feature planned.

            Check the future announcement in the forum.

            Thank again!

            thank you for the detailed howto, if there is traction for this feature maybe it worth documenting officially, I'll ask product managers.

            F 1 Reply Last reply
            Reply Quote 1
            • F
              FritzGerald @rzr
              last edited by

              @rzr Great. Thank you!

              rzrR 1 Reply Last reply
              Reply Quote 1
              • rzrR rzr referenced this topic on
              • rzrR
                rzr Vates 🪐 XCP-ng Team @FritzGerald
                last edited by

                Package landed in testing repo please check the related announcement:

                https://xcp-ng.org/forum/topic/9964/xcp-ng-8-3-updates-announcements-and-testing/432

                Then please confirm the issue is gone by editing post topic with "[SOLVED]" prefix

                1 Reply Last reply
                Reply Quote 1
                • C
                  cobordism
                  last edited by

                  Just for the record, I get the same snmp errors with OpenIPMI instead of nut.

                  # yum update --disablerepo=* --enablerepo=xcp-ng-base,xcp-ng-updates 
                  Loaded plugins: fastestmirror
                  Loading mirror speeds from cached hostfile
                  Excluding mirror: updates.xcp-ng.org
                   * xcp-ng-base: mirrors.xcp-ng.org
                  Excluding mirror: updates.xcp-ng.org
                   * xcp-ng-updates: mirrors.xcp-ng.org
                  Resolving Dependencies
                  --> Running transaction check
                  ---> Package net-snmp.x86_64 1:5.7.2-52.1.xcpng8.3 will be updated
                  ---> Package net-snmp.x86_64 1:5.9.3-8.1.xcpng8.3 will be an update
                  ---> Package net-snmp-agent-libs.x86_64 1:5.7.2-52.1.xcpng8.3 will be updated
                  ---> Package net-snmp-agent-libs.x86_64 1:5.9.3-8.1.xcpng8.3 will be an update
                  ---> Package net-snmp-libs.x86_64 1:5.7.2-52.1.xcpng8.3 will be updated
                  --> Processing Dependency: libnetsnmp.so.31()(64bit) for package: OpenIPMI-2.0.23-2.el7.x86_64
                  ---> Package net-snmp-libs.x86_64 1:5.9.3-8.1.xcpng8.3 will be an update
                  --> Finished Dependency Resolution
                  Error: Package: OpenIPMI-2.0.23-2.el7.x86_64 (@xcp-ng-base)
                             Requires: libnetsnmp.so.31()(64bit)
                             Removing: 1:net-snmp-libs-5.7.2-52.1.xcpng8.3.x86_64 (@xcp-ng-updates)
                                 libnetsnmp.so.31()(64bit)
                             Updated By: 1:net-snmp-libs-5.9.3-8.1.xcpng8.3.x86_64 (xcp-ng-updates)
                                ~libnetsnmp.so.40()(64bit)
                             Available: 1:net-snmp-libs-5.7.2-33.el7_5.2.x86_64 (xcp-ng-base)
                                 libnetsnmp.so.31()(64bit)
                             Available: 1:net-snmp-libs-5.7.2-51.1.xcpng8.3.x86_64 (xcp-ng-base)
                                 libnetsnmp.so.31()(64bit)
                             Available: 1:net-snmp-libs-5.7.2-51.2.xcpng8.3.x86_64 (xcp-ng-base)
                                 libnetsnmp.so.31()(64bit)
                             Available: 1:net-snmp-libs-5.7.2-51.3.xcpng8.3.x86_64 (xcp-ng-base)
                                 libnetsnmp.so.31()(64bit)
                  
                  rzrR 1 Reply Last reply
                  Reply Quote 0
                  • rzrR
                    rzr Vates 🪐 XCP-ng Team @cobordism
                    last edited by rzr

                    @cobordism said:

                    yum update --disablerepo=* --enablerepo=xcp-ng-base,xcp-ng-updates

                    It's currently in testing and will move to updates if everything (not only nut) is ok:

                    yum install --disablerepo=* \
                       --enablerepo=xcp-ng-base,xcp-ng-updates,xcp-ng-testing  nut
                    
                    1 Reply Last reply
                    Reply Quote 1
                    • F
                      FritzGerald
                      last edited by

                      Hi, I just wanted to comment that the provided packages work for all my server. Thank you!

                      1 Reply Last reply
                      Reply Quote 4
                      • K
                        Kajetan321 @FritzGerald
                        last edited by

                        @FritzGerald Thank you for taking the time to write up the install procedure. I was able to follow it on my end and it seems to be working. What commands do you use to set the three services to start up automatically at boot up? Right now if I reboot and run upsc apc@localhost it throws an error until I manually do:

                        upsdrvctl start
                        systemctl start nut-server.service
                        systemctl start nut-monitor.service
                        
                        F 1 Reply Last reply
                        Reply Quote 0
                        • F
                          FritzGerald @Kajetan321
                          last edited by

                          @Kajetan321 Hi, sorry. I am away from office and I can't tell you that off the top of my head right now. I get back to you during the week.

                          F 1 Reply Last reply
                          Reply Quote 0
                          • F
                            FritzGerald @FritzGerald
                            last edited by FritzGerald

                            Hi @kajetan321, it has been a little while when setting this up. And I have to admit, that I am not sure weather the upcoming instructions are complete. So please take it as an indication:

                            #################################
                            Add Service for ups-driver
                            #################################

                            nano /etc/systemd/system/ups-driver.service
                            
                            [Unit]
                            Description=NUT UPS Driver
                            After=network-online.target
                            Wants=network-online.target
                            Before=nut-server.service
                            
                            [Service]
                            Type=oneshot
                            ExecStart=/usr/sbin/upsdrvctl start
                            ExecStop=/usr/sbin/upsdrvctl stop
                            RemainAfterExit=yes
                            
                            [Install]
                            WantedBy=multi-user.target
                            

                            ###################################
                            Add Requirement for nut-server
                            #################################

                            mkdir -p /etc/systemd/system/nut-server.service.d
                            nano /etc/systemd/system/nut-server.service.d/ups-driver.conf
                            
                            [Unit]
                            Requires=ups-driver.service
                            After=ups-driver.service
                            

                            ###################################
                            Add Requirement for nut monitor
                            #################################

                            mkdir -p /etc/systemd/system/nut-monitor.service.d
                            nano /etc/systemd/system/nut-monitor.service.d/nut-server.conf
                            
                            [Unit]
                            Requires=nut-server.service
                            After=nut-server.service
                            

                            ###################################
                            Enable services
                            ###################################

                            systemctl daemon-reload
                            systemctl enable ups-driver.service
                            systemctl enable nut-server.service
                            systemctl enable nut-monitor.service
                            
                            systemctl start nut-monitor.service
                            

                            ###################################
                            Anaylze tools
                            ###################################

                            systemctl list-dependencies nut-monitor.service
                            systemd-analyze critical-chain nut-monitor.service
                            systemctl show nut-monitor.service -p Requires -p Wants -p After
                            systemctl show nut-server.service -p Requires -p Wants -p After
                            

                            Please let me know if it works, since I then would update my documentation above

                            K 2 Replies Last reply
                            Reply Quote 0
                            • K
                              Kajetan321 @FritzGerald
                              last edited by

                              @FritzGerald Understood. Thanks for posting. I will get back to this as soon as I have some time.

                              1 Reply Last reply
                              Reply Quote 0
                              • K
                                Kajetan321 @FritzGerald
                                last edited by Kajetan321

                                @FritzGerald I followed the procedure outline above. I did not see any errors when following the procedure however when I rebooted and then ran "upsc apc@localhost" I got "Error: Connection failure: Cannot assign requested address"

                                When I run "sudo systemctl status nut-server" I get " Active: inactive (dead)"

                                Running

                                upsdrvctl start
                                systemctl start nut-server.service
                                systemctl start nut-monitor.service
                                

                                fixes the issue. It looks like nut-server still does not start automatically at boot. How would I troubleshoot this further? I'm not a Linux system admin 😞

                                Gemini suggested rebooting and running "journalctl -u nut-server.service -b 0" which resulted in "-- No entries --"

                                F 1 Reply Last reply
                                Reply Quote 0
                                • F
                                  FritzGerald @Kajetan321
                                  last edited by FritzGerald

                                  @Kajetan321

                                  I looked into my server and I think I forget some points.

                                  First and foremost I have to say, modifying the systemd files can make you system unbootable! It should not happen, but there is chance since we are also modifying some "natively shipped files". So an update can break your working configuration again and that is not the fault of XCP-NG!!!

                                  That said I assume that after a reboot (prior to any of any input of yours) if you enter:

                                  systemctl status ups-driver.service nut-server.service nut-monitor.service
                                  

                                  you will see that "ups-driver.service" is loaded and active, but the 2 other services are not. If thats the case I think I can help you since it is related to a dependency problem during boot up. Please do the following:

                                  Remove "nut-driver.target" dependency from "nut-server.service" by calling

                                  nano /lib/systemd/system/nut-server.service
                                  

                                  and remove nut-driver.target from routine. I prefer to duplicate and comment the to be modified rows to keep the original code. It then should look like this. DO NOT COPY AND PASTE, read, compare and modify carefully!

                                  
                                  [Unit]
                                  Description=Network UPS Tools - power devices information server
                                  #After=local-fs.target network.target nut-driver.target
                                  After=local-fs.target network.target
                                  # We don't Require drivers to be successfully started! This would be
                                  # a change of behavior compared to init SysV, and could prevent from
                                  # accessing successfully started, at least to audit a system.
                                  #Wants=nut-driver.target
                                  Wants=
                                  # The `upsd` is a networked service (even if bound to a `localhost`)
                                  # so it requires that the OS has some notion of networking already.
                                  # Extending the unit does not require *this* file to be edited, you
                                  # can instead drop in an additional piece of configuration, e.g. add
                                  # a `/etc/systemd/system/nut-server.service.d/network.conf` with:
                                  #   [Unit]
                                  #   Requires=network-online.target
                                  #   After=network-online.target
                                  Requires=network.target
                                  Before=nut-monitor.service
                                  PartOf=nut.target
                                  
                                  [Service]
                                  EnvironmentFile=-/etc/ups/nut.conf
                                  SyslogIdentifier=%N
                                  # Note: foreground mode by default skips writing a PID file (and
                                  # needs Type=simple); can use "-FF" here to create one anyway:
                                  ExecStart=/usr/sbin/upsd -F
                                  ExecReload=/usr/sbin/upsd -c reload -P $MAINPID
                                  
                                  [Install]
                                  WantedBy=nut.target
                                  

                                  Afterwards enable nut.target:

                                  systemctl enable nut.target
                                  

                                  Then call:

                                  systemctl daemon-reload
                                  systemctl start nut.target
                                  systemctl start nut-monitor.service
                                  

                                  Now it should work even after reboot

                                  K 1 Reply Last reply
                                  Reply Quote 1
                                  • poddingueP poddingue marked this topic as a question
                                  • K
                                    Kajetan321 @FritzGerald
                                    last edited by

                                    @FritzGerald Thank you for the follow up. It looks to me like none of the three services are running after a reboot.

                                    Executing

                                    systemctl status ups-driver.service nut-server.service nut-monitor.service
                                    

                                    Results in:

                                    ● ups-driver.service - NUT UPS Driver
                                       Loaded: loaded (/etc/systemd/system/ups-driver.service; disabled; vendor preset: disabled)
                                       Active: inactive (dead)
                                    
                                    ● nut-server.service - Network UPS Tools - power devices information server
                                       Loaded: loaded (/usr/lib/systemd/system/nut-server.service; enabled; vendor preset: disabled)
                                      Drop-In: /etc/systemd/system/nut-server.service.d
                                               └─ups-driver.conf
                                       Active: inactive (dead)
                                    
                                    ● nut-monitor.service - Network UPS Tools - power device monitor and shutdown controller
                                       Loaded: loaded (/usr/lib/systemd/system/nut-monitor.service; enabled; vendor preset: disabled)
                                      Drop-In: /etc/systemd/system/nut-monitor.service.d
                                               └─nut-server.conf
                                       Active: inactive (dead)
                                    
                                    F 1 Reply Last reply
                                    Reply Quote 1
                                    • F
                                      FritzGerald @Kajetan321
                                      last edited by

                                      @Kajetan321
                                      I think you forgot

                                      systemctl enable --now ups-driver.service
                                      
                                      K 1 Reply Last reply
                                      Reply Quote 1
                                      • K
                                        Kajetan321 @FritzGerald
                                        last edited by

                                        @FritzGerald

                                        Thanks! Running

                                        systemctl enable --now ups-driver.service
                                        

                                        got ups-driver.service going, then following the instructions you provided earlier got the rest of the two services running, even after a reboot.

                                        I'm now able to run upsc apc@localhost and get the correct output listing all the properties of the UPS even after a reboot.

                                        F 1 Reply Last reply
                                        Reply Quote 0
                                        • olivierlambertO olivierlambert has marked this topic as solved
                                        • F
                                          FritzGerald @Kajetan321
                                          last edited by

                                          @Kajetan321 Thank you for the feedback. I will update the sections above to ease installation for others

                                          1 Reply Last reply
                                          Reply Quote 0
                                          • F
                                            FritzGerald
                                            last edited by

                                            So, for everyone who wants to install NUT as a service follow these steps. Modifying the systemd files can make you system unbootable! It should not happen, but there is chance since we are also modifying some "natively shipped files". So an update can break your working configuration again and that is not the fault of XCP-NG!!!

                                            As prerequisite it requires the installation and testing of this step-by-step guide https://xcp-ng.org/forum/post/104253. Make sure everything runs before proceeding to the following steps! Now:

                                            #################################
                                            Remove "nut-driver.target" dependency
                                            #################################
                                            Remove "nut-driver.target" dependency from "nut-server.service" by calling

                                            nano /lib/systemd/system/nut-server.service
                                            

                                            and remove nut-driver.target from routine. I prefer to duplicate and comment to keep the modified rows in the original code. It then should look like this. DO NOT COPY AND PASTE, read, compare and modify carefully!

                                            [Unit]
                                            Description=Network UPS Tools - power devices information server
                                            #After=local-fs.target network.target nut-driver.target
                                            After=local-fs.target network.target
                                            # We don't Require drivers to be successfully started! This would be
                                            # a change of behavior compared to init SysV, and could prevent from
                                            # accessing successfully started, at least to audit a system.
                                            #Wants=nut-driver.target
                                            Wants=
                                            # The `upsd` is a networked service (even if bound to a `localhost`)
                                            # so it requires that the OS has some notion of networking already.
                                            # Extending the unit does not require *this* file to be edited, you
                                            # can instead drop in an additional piece of configuration, e.g. add
                                            # a `/etc/systemd/system/nut-server.service.d/network.conf` with:
                                            #   [Unit]
                                            #   Requires=network-online.target
                                            #   After=network-online.target
                                            Requires=network.target
                                            Before=nut-monitor.service
                                            PartOf=nut.target
                                            
                                            [Service]
                                            EnvironmentFile=-/etc/ups/nut.conf
                                            SyslogIdentifier=%N
                                            # Note: foreground mode by default skips writing a PID file (and
                                            # needs Type=simple); can use "-FF" here to create one anyway:
                                            ExecStart=/usr/sbin/upsd -F
                                            ExecReload=/usr/sbin/upsd -c reload -P $MAINPID
                                            
                                            [Install]
                                            WantedBy=nut.target
                                            

                                            #################################
                                            Add Service for ups-driver
                                            #################################

                                            nano /etc/systemd/system/ups-driver.service
                                            
                                            [Unit]
                                            Description=NUT UPS Driver
                                            After=network-online.target
                                            Wants=network-online.target
                                            Before=nut-server.service
                                            
                                            [Service]
                                            Type=oneshot
                                            ExecStart=/usr/sbin/upsdrvctl start
                                            ExecStop=/usr/sbin/upsdrvctl stop
                                            RemainAfterExit=yes
                                            
                                            [Install]
                                            WantedBy=multi-user.target
                                            

                                            ###################################
                                            Add Requirement for nut-server
                                            #################################

                                            mkdir -p /etc/systemd/system/nut-server.service.d
                                            nano /etc/systemd/system/nut-server.service.d/ups-driver.conf
                                            
                                            [Unit]
                                            Requires=ups-driver.service
                                            After=ups-driver.service
                                            

                                            ###################################
                                            Add Requirement for nut monitor
                                            #################################

                                            mkdir -p /etc/systemd/system/nut-monitor.service.d
                                            nano /etc/systemd/system/nut-monitor.service.d/nut-server.conf
                                            
                                            [Unit]
                                            Requires=nut-server.service
                                            After=nut-server.service
                                            

                                            ###################################
                                            Enable services
                                            ###################################

                                            systemctl daemon-reload
                                            systemctl enable ups-driver.service
                                            systemctl enable nut.target
                                            systemctl enable nut-server.service
                                            systemctl enable nut-monitor.service
                                            
                                            systemctl start nut-monitor.service
                                            

                                            ###################################
                                            Analyze & debugging commands
                                            ###################################

                                            systemctl list-dependencies nut-monitor.service
                                            systemd-analyze critical-chain nut-monitor.service
                                            systemctl show nut-monitor.service -p Requires -p Wants -p After
                                            systemctl show nut-server.service -p Requires -p Wants -p After
                                            
                                            1 Reply Last reply
                                            Reply Quote 0

                                            Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                            Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                            With your input, this post could be even better 💗

                                            Register Login
                                            • First post
                                              Last post