Netdata package is now available in XCP-ng

stormi

If you installed netdata-ui, each instance of netdata is independant and has no knowledge of the pool setup. There's no obvious reason why it works on one and not on the other. Except if you installed netdata only instead of netdata-ui.

stormi

What's the output of iptables -L on both hosts?

PC_123

@stormi
Not working host:

[10:07 xcp-ng3 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_input_chain  tcp  --  anywhere             anywhere             tcp dpt:nbd
ACCEPT     gre  --  anywhere             anywhere
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_output_chain  tcp  --  anywhere             anywhere             tcp spt:nbd

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere             icmp any
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:ha-cluster
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:21064
ACCEPT     udp  --  anywhere             anywhere             multiport dports hpoms-dps-lstn,netsupport
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain xapi_nbd_input_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain xapi_nbd_output_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Working Host:

[10:28 xcp-ng2 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_input_chain  tcp  --  anywhere             anywhere             tcp dpt:nbd
ACCEPT     gre  --  anywhere             anywhere
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_output_chain  tcp  --  anywhere             anywhere             tcp spt:nbd

Chain NETDATA (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:dnp-sec

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere             icmp any
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:ha-cluster
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:21064
ACCEPT     udp  --  anywhere             anywhere             multiport dports hpoms-dps-lstn,netsupport
NETDATA    all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain xapi_nbd_input_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain xapi_nbd_output_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

stormi

The firewall rule for netdata is clearly missing here on the first host. I don't know why.

stormi

What's the output of: rpm -qa | grep netdata on both hosts?

PC_123

@stormi

[10:28 xcp-ng3 ~]# rpm -qa | grep netdata
netdata-1.19.0-3.xcpng8.1.x86_64
netdata-ui-1.19.0-3.xcpng8.1.x86_64

[10:29 xcp-ng2 ~]# rpm -qa | grep netdata
netdata-1.19.0-3.xcpng8.1.x86_64
netdata-ui-1.19.0-3.xcpng8.1.x86_64

PC_123

@stormi

I'm glad you were able to spot a difference. My untrained eye still doesn't see the difference.

stormi

Can you run /usr/libexec/netdata/xcpng-iptables-restore.sh on the host where it's not working and share the output?

PC_123

@stormi

That fixed it, thank you. Any idea why the firewall rule wasn't initially created?

[10:59 xcp-ng3 ~]# /usr/libexec/netdata/xcpng-iptables-restore.sh
Applying firewall rules for netdata from /etc/sysconfig/iptables_netdata

stormi

No, I don't know. The iptables rules are rebuilt each time your host starts, and the additional rules for netdata are added when the netdata service starts.

PC_123

@stormi

I just realized your icon is from Keen4. I use to play that game when I was much younger.

Thanks again for all your help!

PC_123

@PC_123
For anyone who has this issue in the future.

The command stormi provided did fix the problem on a running machine. Unfortunately it did not persist following a reboot.

The command: iptables -F

Seems to be a better long term solution.

Thanks again to @stormi for isolating the issue.

stormi

That command I provided is run each time the netdata service starts. So the proper fix is to find out why it doesn't work. Maybe it runs too soon and a dependency needs to be added to another systemd unit.

iptables -F is NOT a fix nor a "better long term solution". It simply deletes all the iptables rules, so basically you're disabling the firewall. By the way it does not survive a reboot.

thisisbenwoo

@olivierlambert I'm experiencing something similar. I recently installed XCP-ng, and the trial Premium license. But it also gives me the netdata not available error. I also don't think I have a xapi0:

uuid ( RO)                : 35124aac-2b87-4fff-8bc7-44ac346437fa
          name-label ( RW): Host internal management network
    name-description ( RW): Network on which guests will be assigned a private link-local IP address which can be used to talk XenAPI
              bridge ( RO): xenapi


uuid ( RO)                : e2e0991c-17a1-22ce-1b3c-ba5178105b7a
          name-label ( RW): Pool-wide network associated with eth4
    name-description ( RW):
              bridge ( RO): xenbr4


uuid ( RO)                : 63d1ff98-833c-f39f-cc65-d5843b20b037
          name-label ( RW): Pool-wide network associated with eth3
    name-description ( RW):
              bridge ( RO): xenbr3


uuid ( RO)                : 8b649421-9686-d09c-19f0-8726c04f9134
          name-label ( RW): Pool-wide network associated with eth0
    name-description ( RW):
              bridge ( RO): xenbr0


uuid ( RO)                : 8c47afa9-3bc0-dec0-3c2e-cd304d28b064
          name-label ( RW): Pool-wide network associated with eth2
    name-description ( RW):
              bridge ( RO): xenbr2


uuid ( RO)                : 37e022e3-d5dd-e0ea-0be6-e2f07d6c75fc
          name-label ( RW): Pool-wide network associated with eth1
    name-description ( RW):
              bridge ( RO): xenbr1

I would prefer to use the integrated installation, so that I can be assured of compatibility, etc. Thanks in advance. The product is awesome!

Professor-X-Tech

@stormi @olivierlambert

Trying to install it on my xcp-ng host and it having issue with the mirrors.

Do we need to do anything else?

stormi

You had the same issue with two different mirrors according to the message, so I'd first look at network issues or configuration.

Professor-X-Tech

@stormi

Just tired downloading one of the package from my other computer and it worked. issue seem to only be with my xcp-ng host. not sure how to troubleshoot this.

stormi

I can't help right now myself but I'm sure other users will have ideas. Can you open a separate thread since it's not directly related to netdata?

Professor-X-Tech

@stormi

Edit:
Thank you! Got it working and looking good.

Troubleshooting non related to Netdata.
I remember I change my upstream DNS to Quad 9 recently (1 week ago) and I just change it to Cloudflair (1.1.1.1) and it was the problem.

Look like Quad 9 (9.9.9.9) does not like the Mirrors for some reason.

discostur

@olivierlambert Just installed netdata package on our xcp-ng cluster. Its running like expected but i can only get ram / memory metrics from the dom0. Is there any way to get the full ram metrics from all VMs / host?

Thanks