Posts made by PC_123 | XCP-ng and XO forum

PC_123

No luck, I've tried building from source, using the script, etc.

Following along with Google Gemini I even managed to get walked into the process of damaging the Xapi to the point of losing access to the server from Xen Orchestra / XCP-ng Center (unfortunately for me it was the Pool Master).

Thankfully I had backups on my NAS and experienced minimal loses through the process of having to reinstall the operating system.

At this point, I'm guessing my pool that dates back to version 8.1 is screwed up in some way beyond human or AI understanding.

I have 2 Servers & 3 NAS units (2 TrueNas Scale) + 1 Synology that all work with the old version of Xen Orchestra but I can't get it to work with anything else. Manually putting the same info in or importing the configuration.

One of the servers was rebuilt today and added back to the pool. Is it possible that reinstalling the OS on the other server could fix everything?

I'm confused because the backups aren't failing from the old Xen Orchestra.

Old:

New:

Any help and / or ideas would be greatly appreciated.

Thanks!

PC_123

@olivierlambert

Good morning Olivier! I believe the issue is with the Xen Orchestrator Installer Updater script no longer adding all of the required services.

I had to do some finagling last night to get the service to even start and become active. By default there were errors everywhere and it wasn't even pointing to the correct location to access the files, yarn, etc.

I'm going to try compiling the application with the documentation you and the team provided manually. I'm sure that was the issue. I'll report back shortly.

Thank you everyone for taking a look and providing guidance!

PC_123

This is extremely strange.

Using a new install (not using the config) I created a link to the Remote NFS Server.

The test works and shows the correct amount of data and a read/write speed so I do believe its connecting properly.

Unfortunately when I run the backup job it fails and the log is showing that the connection failed to reach 127.0.0.1 which isn't the correct address or referenced in the configuration. Does anyone have an idea as to why this is happening to a brand new install of xen orchestra?

PC_123

@Andrew

I tried Debian 11 and it failed during the installation process so I also tried Ubuntu 24.04. It installed successfully but I've received the same error as the one I experienced on Debian 12.

The one things that is really strange is when I export the Xen Orchestra Configuration from my working Ubuntu 18.04 instance into a newly installed instance or Xen Orchestra running on any other version of Linux it adds old remotes at different IP Addresses then what is showing on the working instance of Ubuntu.

It seems that Xen Orchestra is pulling from a database on my pool with incorrect information. Any idea where I can look to find and fix this issue?

PC_123

I'm out of ideas and the my normal helpers (AI Chatbots seem to be out of them too).

I've been a long time XCP-ng user and have an instance of Xen Orchestra built from sources that's been and still runs on Ubuntu 18. Every time I try to update it or my instance of Xen Orchestra is seems to break. With the holiday weekend here, I decided to give it another try and have once again run out of ideas on how to get this working.

On a fresh install of Debian 12, I am trying to get a backup to my remote (TrueNAS Scale) to accept the backups. From my very old Ubuntu 18, the backup jobs work to both TrueNAS and Synology NAS targets. From the fresh install of Debian I can't get of the remotes working. The error I get is:

backupNg.runJob
{
  "id": "4345ad59-46b6-4e19-9d0d-2bab9b75745a",
  "schedule": "8e6198de-8f95-4768-8e39-d17ce8336fcc"
}
{
  "call": {
    "duration": 34,
    "method": "session.login_with_password",
    "params": "* obfuscated *"
  }
}

Does anyone know what might be causing this? I've tried using the same exact config, no config, making a new dataset on the target. Nothing seems to be working. I'd be grateful for any guidance.

I'm sure I missed some things please let me know if I can provide any additional useful information.

If you know of any useful documentation or youtube videos please send them my way. I'm having a hard time understanding why it works on the older version of Ubuntu vs a more modern OS. I'm sure its increased security somewhere.

Thanks for reading, hope someone has seen this before.

PC_123

@adzmedia

One thing to try is the boot mode when creating the VM. The template for Windows defaults to UEFI as the boot mode.

I believe if you use BIOS Boot it will allow you to load the ISO.

Hope that helps!!!

PC_123

@techjeff

Thanks TechJeff!

Yes I have 5 different instances of Xen Orchestra sleeping across my pool. I updated my Primary & Secondary to the latest Xen Orchestra and that's when they stopped working.

Jobs that Xen Orchestra was coordinating for me included booting, backups, and shutdown's.
They all started failing on the new version:

XO-Server 5.83.0 / XO-Web 5.89.0 / Ubuntu 20

The older version running on Ubuntu 18 is XO-Server 5.78.2 / XO-Web 5.80.0 is back to being Primary and works flawlessly so far.

So I'll follow your advice and live on the old version until the issue is resolved.

This is still great software, sometimes it takes these issues to fully appreciate how useful & reliable this application is.

Big Thanks to you super smart people who keep these things working for us!!!

PC_123

I recently needed to rebuild my instance of Xen Orchestra as part of an upgrade.

For some reason when using the Job that triggers host.start it returns with an invalid parameter error.

Any idea on what I could try to resolve this issue?

Error Details:

host.start
{
  "id": [
    "efa4ef5e...."
  ]
}
{
  "code": 10,
  "data": {
    "errors": [
      {
        "code": null,
        "reason": "type",
        "message": "must be string, but is array",
        "property": "@.id"
      }
    ]
  },
  "message": "invalid parameters",
  "name": "XoError",
  "stack": "XoError: invalid parameters
    at Module.invalidParameters (/opt/xo/xo-builds/xen-orchestra-202111080620/packages/xo-common/src/api-errors.js:21:32)
    at Object.call (file:///opt/xo/xo-builds/xen-orchestra-202111080620/packages/xo-server/src/xo-mixins/api.mjs:73:18)
    at Api.callApiMethod (file:///opt/xo/xo-builds/xen-orchestra-202111080620/packages/xo-server/src/xo-mixins/api.mjs:300:19)"
}

PC_123

@PC_123
For anyone who has this issue in the future.

The command stormi provided did fix the problem on a running machine. Unfortunately it did not persist following a reboot.

The command: iptables -F

Seems to be a better long term solution.

Thanks again to @stormi for isolating the issue.

PC_123

@stormi

I just realized your icon is from Keen4. I use to play that game when I was much younger.

Thanks again for all your help!

PC_123

@stormi

That fixed it, thank you. Any idea why the firewall rule wasn't initially created?

[10:59 xcp-ng3 ~]# /usr/libexec/netdata/xcpng-iptables-restore.sh
Applying firewall rules for netdata from /etc/sysconfig/iptables_netdata

PC_123

@stormi

I'm glad you were able to spot a difference. My untrained eye still doesn't see the difference.

PC_123

@stormi

[10:28 xcp-ng3 ~]# rpm -qa | grep netdata
netdata-1.19.0-3.xcpng8.1.x86_64
netdata-ui-1.19.0-3.xcpng8.1.x86_64

[10:29 xcp-ng2 ~]# rpm -qa | grep netdata
netdata-1.19.0-3.xcpng8.1.x86_64
netdata-ui-1.19.0-3.xcpng8.1.x86_64

PC_123

@stormi
Not working host:

[10:07 xcp-ng3 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_input_chain  tcp  --  anywhere             anywhere             tcp dpt:nbd
ACCEPT     gre  --  anywhere             anywhere
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_output_chain  tcp  --  anywhere             anywhere             tcp spt:nbd

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere             icmp any
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:ha-cluster
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:21064
ACCEPT     udp  --  anywhere             anywhere             multiport dports hpoms-dps-lstn,netsupport
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain xapi_nbd_input_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain xapi_nbd_output_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Working Host:

[10:28 xcp-ng2 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_input_chain  tcp  --  anywhere             anywhere             tcp dpt:nbd
ACCEPT     gre  --  anywhere             anywhere
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
xapi_nbd_output_chain  tcp  --  anywhere             anywhere             tcp spt:nbd

Chain NETDATA (1 references)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:dnp-sec

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere             icmp any
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     udp  --  anywhere             anywhere             ctstate NEW udp dpt:ha-cluster
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:ssh
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             ctstate NEW tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:21064
ACCEPT     udp  --  anywhere             anywhere             multiport dports hpoms-dps-lstn,netsupport
NETDATA    all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain xapi_nbd_input_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

Chain xapi_nbd_output_chain (1 references)
target     prot opt source               destination
REJECT     all  --  anywhere             anywhere             reject-with icmp-port-unreachable

PC_123

@PC_123

I ran identical code on two machines. One worked and the other didn't. The machine that didn't work is not the master of the pool. Could that be the reason? Do I need to setup the centralized reporting that was mentioned at the beginning of this thread?

PC_123

@stormi
Green is a good.

[10:06 xcp-ng3 ~]# systemctl status netdata.service
● netdata.service - Real time performance monitoring
   Loaded: loaded (/usr/lib/systemd/system/netdata.service; enabled; vendor pres                                                        et: disabled)
   Active: active (running) since Tue 2020-06-16 09:48:42 EDT; 18min ago
  Process: 1588 ExecStartPre=/usr/libexec/netdata/xcpng-iptables-restore.sh (cod                                                        e=exited, status=0/SUCCESS)
  Process: 1523 ExecStartPre=/bin/chown -R netdata:netdata /var/run/netdata (cod                                                        e=exited, status=0/SUCCESS)
  Process: 1470 ExecStartPre=/bin/mkdir -p /var/run/netdata (code=exited, status                                                        =0/SUCCESS)
  Process: 1404 ExecStartPre=/bin/chown -R netdata:netdata /var/cache/netdata (c                                                        ode=exited, status=0/SUCCESS)
  Process: 1359 ExecStartPre=/bin/mkdir -p /var/cache/netdata (code=exited, stat                                                        us=0/SUCCESS)
 Main PID: 1861 (netdata)
   CGroup: /system.slice/netdata.service
           ├─1861 /usr/sbin/netdata -P /var/run/netdata/netdata.pid -D -W set...
           ├─2035 /usr/libexec/netdata/plugins.d/apps.plugin 1
           ├─2058 /usr/libexec/netdata/plugins.d/go.d.plugin 1
           ├─2063 /usr/libexec/netdata/plugins.d/xenstat.plugin 1
           └─2069 /usr/bin/python /usr/libexec/netdata/plugins.d/python.d.plu...

PC_123

@olivierlambert

Imagine that I don't know what to do. I've tried reviewing the Netdata documentation but it isn't specific to XCP-NG.

I ran the below code and received a response. Does that indicate where the issue could be? Also...what code do you suggest I run to check to see if Netdata is running? Why can't it bind to the host address?

[09:49 xcp-ng3 ~]# /usr/sbin/netdata
2020-06-16 09:49:55: netdata INFO  : MAIN : SIGNAL: Not enabling reaper
2020-06-16 09:49:55: netdata ERROR : MAIN : LISTENER: IPv4 bind() on ip '0.0.0.0' port 19999, socktype 1 failed. (errno 98, Address already in use)
2020-06-16 09:49:55: netdata ERROR : MAIN : LISTENER: Cannot bind to ip '0.0.0.0', port 19999
2020-06-16 09:49:55: netdata ERROR : MAIN : LISTENER: IPv6 bind() on ip '::' port 19999, socktype 1 failed. (errno 98, Address already in use)
2020-06-16 09:49:55: netdata ERROR : MAIN : LISTENER: Cannot bind to ip '::', port 19999
2020-06-16 09:49:55: netdata FATAL : MAIN :LISTENER: Cannot listen on any API socket. Exiting... # : Success

2020-06-16 09:49:55: netdata INFO  : MAIN : EXIT: netdata prepares to exit with code 1...
2020-06-16 09:49:55: netdata INFO  : MAIN : EXIT: cleaning up the database...
2020-06-16 09:49:55: netdata INFO  : MAIN : Cleaning up database [0 hosts(s)]...
2020-06-16 09:49:55: netdata INFO  : MAIN : EXIT: all done - netdata is now exiting - bye bye...

PC_123

@olivierlambert
After running the command "yum install netdata-ui" on 2 hosts. Only one started working.

I can reach the netdata page on the working server at: url:19999. I've rebooted the newer of the 2 servers and no matter what I've tried the netdata page doesn't appear.

If I try the command again I receive an error indicating that the package is already installed and there is nothing to do.

Is there an easy way to fix this or, what's the command to rollback this install?

Thanks to everyone in the community (especially @olivierlambert) for your hard work on this software.