XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    NBD Backup not working

    Scheduled Pinned Locked Moved Xen Orchestra
    23 Posts 3 Posters 2.8k Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K Offline
      KLSsandman
      last edited by

      Hello everyone,

      I am trying to test NBD backups. I have followed the guide, enabled for the pool network, added the option to the XOA config and restart XOA. I've also enabled the multiple blocks option for my target etc.

      XOA is on a network called MGMT, all hosts have a nic in the MGMT network (which XOA communicates with hosts on) and a nic in the LAN network.

      My testing vm is on another host to XOA. The vm is connected to the LAN network

      My backup target is a physical SMB windows server on the same LAN network segment as the VM. Normal backups work fine, just a massive VHD file gets created still. I tried enabling NBD on MGMT, LAN only and both network pools. I can see the little padlock on the network pools and under the networking tab for the test vm.

      Any idea what I am missing please? Does XCP-Ng host need to be at a specific version also?

      Many thanks

      florentF 1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Hi,

        Are you using XOA or XO from the sources?

        K 1 Reply Last reply Reply Quote 0
        • K Offline
          KLSsandman @olivierlambert
          last edited by

          @olivierlambert thanks for your reply. I'm using XOA 5.78.0

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            So you are on XOA stable release channel, right?

            K 1 Reply Last reply Reply Quote 0
            • K Offline
              KLSsandman @olivierlambert
              last edited by

              @olivierlambert yes, thats correct

              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                Let me ping @florent about this (but right now he's very busy)

                K 1 Reply Last reply Reply Quote 0
                • K Offline
                  KLSsandman @olivierlambert
                  last edited by

                  @olivierlambert thank you

                  1 Reply Last reply Reply Quote 0
                  • florentF Offline
                    florent Vates 🪐 XO Team @KLSsandman
                    last edited by

                    @KLSsandman hi

                    for now there isn't any UI , but there should be info in the XO logs with NBD client ready if everything is ok

                    you can also activate debug on nbd by adding a filter on lthe log section of your config file

                    [logs]
                    # Display all logs matching this filter, regardless of their level
                    filter = 'xo:backups:DeltaBackupWriter'
                    

                    you'll have additionnal informations , if the configuration is loaded correctly and if the connexion to the nbd server works

                    K 1 Reply Last reply Reply Quote 0
                    • K Offline
                      KLSsandman @florent
                      last edited by

                      @florent Thank you for your reply. Would the [Logs] entry go into the config.toml file?

                      florentF 1 Reply Last reply Reply Quote 0
                      • florentF Offline
                        florent Vates 🪐 XO Team @KLSsandman
                        last edited by

                        @KLSsandman yes, or you can create a file next to it with the .toml extension

                        K 1 Reply Last reply Reply Quote 0
                        • K Offline
                          KLSsandman @florent
                          last edited by olivierlambert

                          @florent Hi

                          I've added the [Logs] section to my config file and restarted XOA. I've started a test backup job but don't see anything in the syslog with regards NBD 😞

                          Feb 10 05:46:30 xoa systemd[1354]: run-xo\x2dserver-mounts-324ac22a\x2d4307\x2d4bb2\x2db705\x2df6bb73799a13.mount: Succeeded.
                          Feb 10 05:46:30 xoa kernel: [  447.931697] No dialect specified on mount. Default has changed to a more secure dialect, SMB2.1 or later (e.g. SMB3), from CIFS (SMB1). To use the less secure SMB1 dialect to access old servers which do no$
                          Feb 10 05:46:33 xoa systemd[1]: Stopping User Manager for UID 1001...
                          Feb 10 05:46:33 xoa systemd[2550]: Stopped target Default.
                          Feb 10 05:46:33 xoa systemd[2550]: Stopped target Basic System.
                          Feb 10 05:46:33 xoa systemd[2550]: Stopped target Timers.
                          Feb 10 05:46:33 xoa systemd[2550]: Stopped target Sockets.
                          Feb 10 05:46:33 xoa systemd[2550]: gpg-agent-extra.socket: Succeeded.
                          Feb 10 05:46:33 xoa systemd[2550]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
                          Feb 10 05:46:33 xoa systemd[2550]: dirmngr.socket: Succeeded.
                          Feb 10 05:46:33 xoa systemd[2550]: Closed GnuPG network certificate management daemon.
                          Feb 10 05:46:33 xoa systemd[2550]: gpg-agent.socket: Succeeded.
                          Feb 10 05:46:33 xoa systemd[2550]: Closed GnuPG cryptographic agent and passphrase cache.
                          Feb 10 05:46:33 xoa systemd[2550]: gpg-agent-browser.socket: Succeeded.
                          Feb 10 05:46:33 xoa systemd[2550]: Closed GnuPG cryptographic agent and passphrase cache (access for web browsers).
                          Feb 10 05:46:33 xoa systemd[2550]: gpg-agent-ssh.socket: Succeeded.
                          Feb 10 05:46:33 xoa systemd[2550]: Closed GnuPG cryptographic agent (ssh-agent emulation).
                          Feb 10 05:46:33 xoa systemd[2550]: Stopped target Paths.
                          Feb 10 05:46:33 xoa systemd[2550]: Reached target Shutdown.
                          Feb 10 05:46:33 xoa systemd[2550]: systemd-exit.service: Succeeded.
                          Feb 10 05:46:33 xoa systemd[2550]: Started Exit the Session.
                          Feb 10 05:46:33 xoa systemd[2550]: Reached target Exit the Session.
                          Feb 10 05:46:33 xoa systemd[1]: user@1001.service: Succeeded.
                          Feb 10 05:46:33 xoa systemd[1]: Stopped User Manager for UID 1001.
                          Feb 10 05:46:33 xoa systemd[1]: Stopping User Runtime Directory /run/user/1001...
                          Feb 10 05:46:33 xoa systemd[1354]: run-user-1001.mount: Succeeded.
                          Feb 10 05:46:33 xoa systemd[1]: run-user-1001.mount: Succeeded.
                          Feb 10 05:46:33 xoa systemd[1]: user-runtime-dir@1001.service: Succeeded.
                          Feb 10 05:46:33 xoa systemd[1]: Stopped User Runtime Directory /run/user/1001.
                          Feb 10 05:46:33 xoa systemd[1]: Removed slice User Slice of UID 1001.
                          Feb 10 05:46:36 xoa kernel: [  453.340494] [UFW BLOCK] IN=eth0 OUT= MAC=9e:f2:26:65:70:99:2c:44:fd:95:84:04:08:00 SRC=192.168.220.180 DST=192.168.220.159 LEN=1500 TOS=0x00 PREC=0x00 TTL=64 ID=26082 DF PROTO=TCP SPT=443 DPT=47288 WINDOW=$
                          Feb 10 05:46:41 xoa kernel: [  458.088847] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:46:52 xoa systemd[1]: systemd-timedated.service: Succeeded.
                          Feb 10 05:47:11 xoa kernel: [  488.257006] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:47:41 xoa kernel: [  518.436157] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:48:11 xoa kernel: [  548.623899] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:48:36 xoa kernel: [  573.657843] [UFW BLOCK] IN=eth0 OUT= MAC=9e:f2:26:65:70:99:2c:44:fd:95:84:04:08:00 SRC=192.168.220.180 DST=192.168.220.159 LEN=1500 TOS=0x00 PREC=0x00 TTL=64 ID=26083 DF PROTO=TCP SPT=443 DPT=47288 WINDOW=$
                          Feb 10 05:48:41 xoa kernel: [  578.789296] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:49:11 xoa kernel: [  608.965861] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:49:42 xoa kernel: [  639.131859] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:50:12 xoa kernel: [  669.298617] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:50:36 xoa kernel: [  693.975210] [UFW BLOCK] IN=eth0 OUT= MAC=9e:f2:26:65:70:99:2c:44:fd:95:84:04:08:00 SRC=192.168.220.180 DST=192.168.220.159 LEN=1500 TOS=0x00 PREC=0x00 TTL=64 ID=26084 DF PROTO=TCP SPT=443 DPT=47288 WINDOW=$
                          Feb 10 05:50:42 xoa kernel: [  699.462074] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:51:12 xoa kernel: [  729.643382] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:51:16 xoa ntpd[496]: 80.82.245.120 local addr 192.168.220.159 -> <null>
                          Feb 10 05:51:18 xoa ntpd[496]: 185.244.195.159 local addr 192.168.220.159 -> <null>
                          Feb 10 05:51:33 xoa ntpd[496]: 65.108.59.245 local addr 192.168.220.159 -> <null>
                          Feb 10 05:51:42 xoa kernel: [  759.827172] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:52:13 xoa kernel: [  789.996962] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:52:43 xoa kernel: [  820.164161] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:53:13 xoa kernel: [  850.332900] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:53:43 xoa kernel: [  880.502728] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:54:04 xoa systemd[1]: Starting Cleanup of Temporary Directories...
                          Feb 10 05:54:04 xoa systemd[1]: systemd-tmpfiles-clean.service: Succeeded.
                          Feb 10 05:54:04 xoa systemd[1]: Started Cleanup of Temporary Directories.
                          Feb 10 05:54:13 xoa kernel: [  910.675743] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          Feb 10 05:54:43 xoa kernel: [  940.846494] [UFW BLOCK] IN=eth0 OUT= MAC=33:33:00:00:00:01:10:be:f5:aa:90:e0:86:dd SRC=fe80:0000:0000:0000:12be:f5ff:feaa:90e0 DST=ff02:0000:0000:0000:0000:0000:0000:0001 LEN=377 TC=0 HOPLIMIT=255 FLOWLBL=$
                          
                          florentF 1 Reply Last reply Reply Quote 0
                          • florentF Offline
                            florent Vates 🪐 XO Team @KLSsandman
                            last edited by florent

                            @KLSsandman I don't see anything about nbd or even a backup

                            You can filter the interesting part by using journalctl -u xo-server -f and then launching a backup

                            K 1 Reply Last reply Reply Quote 0
                            • K Offline
                              KLSsandman @florent
                              last edited by olivierlambert

                              @florent said in NBD Backup not working:

                              journalctl -u xo-server -f

                              This is strange, backups running but only entry is:

                              journalctl -u xo-server -f
                              -- Logs begin at Fri 2023-02-10 11:12:03 GMT. --
                              Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                              Feb 13 01:00:01 xoa xo-server[450]:       at addChunk (node:internal/streams/readable:324:12)
                              Feb 13 01:00:01 xoa xo-server[450]:       at readableAddChunk (node:internal/streams/readable:297:9)
                              Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
                              Feb 13 01:00:01 xoa xo-server[450]:       at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)
                              Feb 13 01:00:01 xoa xo-server[450]:       at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                              Feb 13 01:00:01 xoa xo-server[450]:     code: 'EENVELOPE',
                              Feb 13 01:00:01 xoa xo-server[450]:     command: 'API'
                              Feb 13 01:00:01 xoa xo-server[450]:   }
                              Feb 13 01:00:01 xoa xo-server[450]: }
                              

                              Thanks
                              Simon

                              florentF 1 Reply Last reply Reply Quote 0
                              • florentF Offline
                                florent Vates 🪐 XO Team @KLSsandman
                                last edited by

                                @KLSsandman can you get a few more line before ?

                                journalctl -u xo-server -f -n 1000 for example

                                K 2 Replies Last reply Reply Quote 0
                                • K Offline
                                  KLSsandman @florent
                                  last edited by olivierlambert

                                  @florent said in NBD Backup not working:

                                  journalctl -u xo-server -f -n 1000

                                  Bit more in this one, can't see anything relating to NBD though

                                  journalctl -u xo-server -f -n 1000
                                  -- Logs begin at Fri 2023-02-10 11:12:03 GMT. --
                                  Feb 10 11:12:06 xoa systemd[1]: Started XO Server.
                                  Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.778Z xo:main INFO Configuration loaded.
                                  Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.787Z xo:main INFO Web server listening on http://[::]:80
                                  Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.787Z xo:main INFO Web server listening on http://127.0.0.1:38889
                                  Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.824Z xo:main INFO Web server listening on https://[::]:443
                                  Feb 10 11:12:11 xoa xo-server[450]: 2023-02-10T11:12:11.499Z xo:xo-mixins:xen-servers WARN failed to connect to XenServer {
                                  Feb 10 11:12:11 xoa xo-server[450]:   host: '192.168.220.155',
                                  Feb 10 11:12:11 xoa xo-server[450]:   error: Error: connect ECONNREFUSED 192.168.220.155:443
                                  Feb 10 11:12:11 xoa xo-server[450]:       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16)
                                  Feb 10 11:12:11 xoa xo-server[450]:       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                                  Feb 10 11:12:11 xoa xo-server[450]:     errno: -111,
                                  Feb 10 11:12:11 xoa xo-server[450]:     code: 'ECONNREFUSED',
                                  Feb 10 11:12:11 xoa xo-server[450]:     syscall: 'connect',
                                  Feb 10 11:12:11 xoa xo-server[450]:     address: '192.168.220.155',
                                  Feb 10 11:12:11 xoa xo-server[450]:     port: 443,
                                  Feb 10 11:12:11 xoa xo-server[450]:     url: 'https://192.168.220.155/jsonrpc',
                                  Feb 10 11:12:11 xoa xo-server[450]:     call: { method: 'session.login_with_password', params: '* obfuscated *' }
                                  Feb 10 11:12:11 xoa xo-server[450]:   }
                                  Feb 10 11:12:11 xoa xo-server[450]: }
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.012Z xo:main INFO Setting up / → /usr/local/lib/node_modules/xo-web/dist
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.202Z xo:plugin INFO register audit
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.205Z xo:plugin INFO register auth-github
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.208Z xo:plugin INFO register auth-google
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.209Z xo:plugin INFO register auth-ldap
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.210Z xo:plugin INFO register auth-saml
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.212Z xo:plugin INFO register backup-reports
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.215Z xo:plugin INFO register load-balancer
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.217Z xo:plugin INFO register netbox
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.219Z xo:plugin INFO register netdata
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.220Z xo:plugin INFO register perf-alert
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.222Z xo:plugin INFO register sdn-controller
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.225Z xo:plugin INFO register telemetry
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.227Z xo:plugin INFO register transport-email
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.228Z xo:plugin INFO register transport-icinga2
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.231Z xo:plugin INFO register transport-nagios
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.233Z xo:plugin INFO register transport-slack
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.234Z xo:plugin INFO register transport-xmpp
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.236Z xo:plugin INFO register usage-report
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.247Z xo:plugin INFO register web-hooks
                                  Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.250Z xo:plugin INFO register xoa
                                  Feb 10 11:12:15 xoa xo-server[450]: 2023-02-10T11:12:15.675Z xo:xo-mixins:xen-servers WARN failed to connect to XenServer {
                                  Feb 10 11:12:15 xoa xo-server[450]:   host: '192.168.220.158',
                                  Feb 10 11:12:15 xoa xo-server[450]:   error: Error: connect EHOSTUNREACH 192.168.220.158:443
                                  Feb 10 11:12:15 xoa xo-server[450]:       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16)
                                  Feb 10 11:12:15 xoa xo-server[450]:       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                                  Feb 10 11:12:15 xoa xo-server[450]:     errno: -113,
                                  Feb 10 11:12:15 xoa xo-server[450]:     code: 'EHOSTUNREACH',
                                  Feb 10 11:12:15 xoa xo-server[450]:     syscall: 'connect',
                                  Feb 10 11:12:15 xoa xo-server[450]:     address: '192.168.220.158',
                                  Feb 10 11:12:15 xoa xo-server[450]:     port: 443,
                                  Feb 10 11:12:15 xoa xo-server[450]:     url: 'https://192.168.220.158/jsonrpc',
                                  Feb 10 11:12:15 xoa xo-server[450]:     call: { method: 'session.login_with_password', params: '* obfuscated *' }
                                  Feb 10 11:12:15 xoa xo-server[450]:   }
                                  Feb 10 11:12:15 xoa xo-server[450]: }
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.371Z xo:plugin INFO successfully register auth-github
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.371Z xo:plugin INFO successfully register auth-google
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register auth-ldap
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register auth-saml
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register netbox
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-icinga2
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-nagios
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-slack
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-xmpp
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register web-hooks
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register telemetry
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register xoa
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register backup-reports
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register transport-email
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.563Z xo:plugin INFO successfully register audit
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.563Z xo:plugin INFO successfully register load-balancer
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.563Z xo:plugin INFO successfully register perf-alert
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.661Z xo:plugin INFO successfully register usage-report
                                  Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.676Z xo:plugin INFO successfully register sdn-controller
                                  Feb 10 11:12:18 xoa xo-server[450]: 2023-02-10T11:12:18.040Z xo:plugin INFO successfully register netdata
                                  Feb 10 11:13:50 xoa xo-server[450]: 2023-02-10T11:13:50.360Z xo:main WARN WebSocket send: {
                                  Feb 10 11:13:50 xoa xo-server[450]:   error: Error: The socket was closed while data was being compressed
                                  Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/sender.js:410:21
                                  Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:325:9
                                  Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:455:7
                                  Feb 10 11:13:50 xoa xo-server[450]:       at afterWrite (node:internal/streams/writable:500:5)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at onwrite (node:internal/streams/writable:480:7)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.cb (node:internal/streams/transform:190:7)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.processCallback (node:zlib:613:8)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)
                                  Feb 10 11:13:50 xoa xo-server[450]: }
                                  Feb 10 11:13:50 xoa xo-server[450]: 2023-02-10T11:13:50.364Z xo:main WARN WebSocket send: {
                                  Feb 10 11:13:50 xoa xo-server[450]:   error: Error: The socket was closed while data was being compressed
                                  Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/sender.js:410:21
                                  Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:325:9
                                  Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:455:7
                                  Feb 10 11:13:50 xoa xo-server[450]:       at afterWrite (node:internal/streams/writable:500:5)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at onwrite (node:internal/streams/writable:480:7)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.cb (node:internal/streams/transform:190:7)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.processCallback (node:zlib:613:8)
                                  Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)
                                  Feb 10 11:13:50 xoa xo-server[450]: }
                                  Feb 10 11:13:51 xoa su[1629]: (to xoa-support) root on none
                                  Feb 10 11:13:51 xoa su[1629]: pam_unix(su-l:session): session opened for user xoa-support by (uid=0)
                                  Feb 10 11:13:51 xoa su[1629]: pam_unix(su-l:session): session closed for user xoa-support
                                  Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.621Z xo:xo-mixins:backups-ng WARN no such object 6ddd6366-62a5-d4b9-9ed1-c5802b5370d9 {
                                  Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object 6ddd6366-62a5-d4b9-9ed1-c5802b5370d9
                                  Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                  Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                  Feb 13 00:00:00 xoa xo-server[450]:     data: { id: '6ddd6366-62a5-d4b9-9ed1-c5802b5370d9', type: undefined }
                                  Feb 13 00:00:00 xoa xo-server[450]:   }
                                  Feb 13 00:00:00 xoa xo-server[450]: }
                                  Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.778Z xo:xo-mixins:backups-ng WARN no such object 5476cf20-72a2-3837-25ff-3386fbc412b6 {
                                  Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object 5476cf20-72a2-3837-25ff-3386fbc412b6
                                  Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                  Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                  Feb 13 00:00:00 xoa xo-server[450]:     data: { id: '5476cf20-72a2-3837-25ff-3386fbc412b6', type: undefined }
                                  Feb 13 00:00:00 xoa xo-server[450]:   }
                                  Feb 13 00:00:00 xoa xo-server[450]: }
                                  Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.780Z xo:xo-mixins:backups-ng WARN no such object fe407fa9-96c6-0978-81ae-12b8874aa2dc {
                                  Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object fe407fa9-96c6-0978-81ae-12b8874aa2dc
                                  Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                  Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                  Feb 13 00:00:00 xoa xo-server[450]:     data: { id: 'fe407fa9-96c6-0978-81ae-12b8874aa2dc', type: undefined }
                                  Feb 13 00:00:00 xoa xo-server[450]:   }
                                  Feb 13 00:00:00 xoa xo-server[450]: }
                                  Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.781Z xo:xo-mixins:backups-ng WARN no such object 52fd8980-4a3f-9b29-a733-78abb9a603e5 {
                                  Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object 52fd8980-4a3f-9b29-a733-78abb9a603e5
                                  Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                  Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                  Feb 13 00:00:00 xoa xo-server[450]:     data: { id: '52fd8980-4a3f-9b29-a733-78abb9a603e5', type: undefined }
                                  Feb 13 00:00:00 xoa xo-server[450]:   }
                                  Feb 13 00:00:00 xoa xo-server[450]: }
                                  Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.785Z xo:xo-mixins:backups-ng WARN no such object d8e7f553-090d-2b67-acc3-83da0b58b8db {
                                  Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object d8e7f553-090d-2b67-acc3-83da0b58b8db
                                  Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                  Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                  Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                  Feb 13 00:00:00 xoa xo-server[450]:     data: { id: 'd8e7f553-090d-2b67-acc3-83da0b58b8db', type: undefined }
                                  Feb 13 00:00:00 xoa xo-server[450]:   }
                                  Feb 13 00:00:00 xoa xo-server[450]: }
                                  Feb 13 01:00:01 xoa xo-server[450]: [WARN] plugin transport-email: Error: No recipients defined
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._formatError (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:774:19)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._setEnvelope (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:985:34)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.send (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:605:14)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at sendMessage (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:227:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at /usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:288:21
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.<anonymous> (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:209:17)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at Object.onceWrapper (node:events:627:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.emit (node:events:513:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._actionEHLO (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:1303:14)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._processResponse (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:932:20)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._onData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:739:14)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.SMTPConnection._onSocketData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:189:44)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.emit (node:events:513:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at addChunk (node:internal/streams/readable:324:12)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at readableAddChunk (node:internal/streams/readable:297:9)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)
                                  Feb 13 01:00:01 xoa xo-server[450]:     at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17)
                                  Feb 13 01:00:01 xoa xo-server[450]: 2023-02-13T01:00:01.340Z xo:xo-server WARN possibly unhandled rejection {
                                  Feb 13 01:00:01 xoa xo-server[450]:   error: Error: No recipients defined
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._formatError (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:774:19)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._setEnvelope (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:985:34)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.send (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:605:14)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at sendMessage (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:227:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:288:21
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.<anonymous> (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:209:17)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at Object.onceWrapper (node:events:627:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.emit (node:events:513:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._actionEHLO (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:1303:14)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._processResponse (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:932:20)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._onData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:739:14)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.SMTPConnection._onSocketData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:189:44)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.emit (node:events:513:28)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at addChunk (node:internal/streams/readable:324:12)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at readableAddChunk (node:internal/streams/readable:297:9)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)
                                  Feb 13 01:00:01 xoa xo-server[450]:       at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                                  Feb 13 01:00:01 xoa xo-server[450]:     code: 'EENVELOPE',
                                  Feb 13 01:00:01 xoa xo-server[450]:     command: 'API'
                                  Feb 13 01:00:01 xoa xo-server[450]:   }
                                  Feb 13 01:00:01 xoa xo-server[450]: }
                                  

                                  Thanks
                                  Simon

                                  florentF 1 Reply Last reply Reply Quote 0
                                  • florentF Offline
                                    florent Vates 🪐 XO Team @KLSsandman
                                    last edited by

                                    @KLSsandman did the backup un through ? it seems like it was launched before the vm list was loaded

                                    K 1 Reply Last reply Reply Quote 0
                                    • K Offline
                                      KLSsandman @florent
                                      last edited by olivierlambert

                                      @florent there is a backup jopb currently running which will take a while as its about 75 vm's. But I have a small test backup job that I start and finishes in about 10mins but the output of the log is the same. I've just set it going and this is the log output:

                                      journalctl -u xo-server -f -n 1000
                                      -- Logs begin at Fri 2023-02-10 11:12:03 GMT. --
                                      Feb 10 11:12:06 xoa systemd[1]: Started XO Server.
                                      Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.778Z xo:main INFO Configuration loaded.
                                      Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.787Z xo:main INFO Web server listening on http://[::]:80
                                      Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.787Z xo:main INFO Web server listening on http://127.0.0.1:38889
                                      Feb 10 11:12:10 xoa xo-server[450]: 2023-02-10T11:12:10.824Z xo:main INFO Web server listening on https://[::]:443
                                      Feb 10 11:12:11 xoa xo-server[450]: 2023-02-10T11:12:11.499Z xo:xo-mixins:xen-servers WARN failed to connect to XenServer {
                                      Feb 10 11:12:11 xoa xo-server[450]:   host: '192.168.220.155',
                                      Feb 10 11:12:11 xoa xo-server[450]:   error: Error: connect ECONNREFUSED 192.168.220.155:443
                                      Feb 10 11:12:11 xoa xo-server[450]:       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16)
                                      Feb 10 11:12:11 xoa xo-server[450]:       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                                      Feb 10 11:12:11 xoa xo-server[450]:     errno: -111,
                                      Feb 10 11:12:11 xoa xo-server[450]:     code: 'ECONNREFUSED',
                                      Feb 10 11:12:11 xoa xo-server[450]:     syscall: 'connect',
                                      Feb 10 11:12:11 xoa xo-server[450]:     address: '192.168.220.155',
                                      Feb 10 11:12:11 xoa xo-server[450]:     port: 443,
                                      Feb 10 11:12:11 xoa xo-server[450]:     url: 'https://192.168.220.155/jsonrpc',
                                      Feb 10 11:12:11 xoa xo-server[450]:     call: { method: 'session.login_with_password', params: '* obfuscated *' }
                                      Feb 10 11:12:11 xoa xo-server[450]:   }
                                      Feb 10 11:12:11 xoa xo-server[450]: }
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.012Z xo:main INFO Setting up / → /usr/local/lib/node_modules/xo-web/dist
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.202Z xo:plugin INFO register audit
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.205Z xo:plugin INFO register auth-github
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.208Z xo:plugin INFO register auth-google
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.209Z xo:plugin INFO register auth-ldap
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.210Z xo:plugin INFO register auth-saml
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.212Z xo:plugin INFO register backup-reports
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.215Z xo:plugin INFO register load-balancer
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.217Z xo:plugin INFO register netbox
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.219Z xo:plugin INFO register netdata
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.220Z xo:plugin INFO register perf-alert
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.222Z xo:plugin INFO register sdn-controller
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.225Z xo:plugin INFO register telemetry
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.227Z xo:plugin INFO register transport-email
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.228Z xo:plugin INFO register transport-icinga2
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.231Z xo:plugin INFO register transport-nagios
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.233Z xo:plugin INFO register transport-slack
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.234Z xo:plugin INFO register transport-xmpp
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.236Z xo:plugin INFO register usage-report
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.247Z xo:plugin INFO register web-hooks
                                      Feb 10 11:12:14 xoa xo-server[450]: 2023-02-10T11:12:14.250Z xo:plugin INFO register xoa
                                      Feb 10 11:12:15 xoa xo-server[450]: 2023-02-10T11:12:15.675Z xo:xo-mixins:xen-servers WARN failed to connect to XenServer {
                                      Feb 10 11:12:15 xoa xo-server[450]:   host: '192.168.220.158',
                                      Feb 10 11:12:15 xoa xo-server[450]:   error: Error: connect EHOSTUNREACH 192.168.220.158:443
                                      Feb 10 11:12:15 xoa xo-server[450]:       at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1300:16)
                                      Feb 10 11:12:15 xoa xo-server[450]:       at TCPConnectWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                                      Feb 10 11:12:15 xoa xo-server[450]:     errno: -113,
                                      Feb 10 11:12:15 xoa xo-server[450]:     code: 'EHOSTUNREACH',
                                      Feb 10 11:12:15 xoa xo-server[450]:     syscall: 'connect',
                                      Feb 10 11:12:15 xoa xo-server[450]:     address: '192.168.220.158',
                                      Feb 10 11:12:15 xoa xo-server[450]:     port: 443,
                                      Feb 10 11:12:15 xoa xo-server[450]:     url: 'https://192.168.220.158/jsonrpc',
                                      Feb 10 11:12:15 xoa xo-server[450]:     call: { method: 'session.login_with_password', params: '* obfuscated *' }
                                      Feb 10 11:12:15 xoa xo-server[450]:   }
                                      Feb 10 11:12:15 xoa xo-server[450]: }
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.371Z xo:plugin INFO successfully register auth-github
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.371Z xo:plugin INFO successfully register auth-google
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register auth-ldap
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register auth-saml
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register netbox
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-icinga2
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-nagios
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-slack
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.372Z xo:plugin INFO successfully register transport-xmpp
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register web-hooks
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register telemetry
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register xoa
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register backup-reports
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.373Z xo:plugin INFO successfully register transport-email
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.563Z xo:plugin INFO successfully register audit
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.563Z xo:plugin INFO successfully register load-balancer
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.563Z xo:plugin INFO successfully register perf-alert
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.661Z xo:plugin INFO successfully register usage-report
                                      Feb 10 11:12:17 xoa xo-server[450]: 2023-02-10T11:12:17.676Z xo:plugin INFO successfully register sdn-controller
                                      Feb 10 11:12:18 xoa xo-server[450]: 2023-02-10T11:12:18.040Z xo:plugin INFO successfully register netdata
                                      Feb 10 11:13:50 xoa xo-server[450]: 2023-02-10T11:13:50.360Z xo:main WARN WebSocket send: {
                                      Feb 10 11:13:50 xoa xo-server[450]:   error: Error: The socket was closed while data was being compressed
                                      Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/sender.js:410:21
                                      Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:325:9
                                      Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:455:7
                                      Feb 10 11:13:50 xoa xo-server[450]:       at afterWrite (node:internal/streams/writable:500:5)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at onwrite (node:internal/streams/writable:480:7)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.cb (node:internal/streams/transform:190:7)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.processCallback (node:zlib:613:8)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)
                                      Feb 10 11:13:50 xoa xo-server[450]: }
                                      Feb 10 11:13:50 xoa xo-server[450]: 2023-02-10T11:13:50.364Z xo:main WARN WebSocket send: {
                                      Feb 10 11:13:50 xoa xo-server[450]:   error: Error: The socket was closed while data was being compressed
                                      Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/sender.js:410:21
                                      Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:325:9
                                      Feb 10 11:13:50 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server/node_modules/ws/lib/permessage-deflate.js:455:7
                                      Feb 10 11:13:50 xoa xo-server[450]:       at afterWrite (node:internal/streams/writable:500:5)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at onwrite (node:internal/streams/writable:480:7)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.cb (node:internal/streams/transform:190:7)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.processCallback (node:zlib:613:8)
                                      Feb 10 11:13:50 xoa xo-server[450]:       at Zlib.callbackTrampoline (node:internal/async_hooks:130:17)
                                      Feb 10 11:13:50 xoa xo-server[450]: }
                                      Feb 10 11:13:51 xoa su[1629]: (to xoa-support) root on none
                                      Feb 10 11:13:51 xoa su[1629]: pam_unix(su-l:session): session opened for user xoa-support by (uid=0)
                                      Feb 10 11:13:51 xoa su[1629]: pam_unix(su-l:session): session closed for user xoa-support
                                      Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.621Z xo:xo-mixins:backups-ng WARN no such object 6ddd6366-62a5-d4b9-9ed1-c5802b5370d9 {
                                      Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object 6ddd6366-62a5-d4b9-9ed1-c5802b5370d9
                                      Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                      Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                      Feb 13 00:00:00 xoa xo-server[450]:     data: { id: '6ddd6366-62a5-d4b9-9ed1-c5802b5370d9', type: undefined }
                                      Feb 13 00:00:00 xoa xo-server[450]:   }
                                      Feb 13 00:00:00 xoa xo-server[450]: }
                                      Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.778Z xo:xo-mixins:backups-ng WARN no such object 5476cf20-72a2-3837-25ff-3386fbc412b6 {
                                      Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object 5476cf20-72a2-3837-25ff-3386fbc412b6
                                      Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                      Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                      Feb 13 00:00:00 xoa xo-server[450]:     data: { id: '5476cf20-72a2-3837-25ff-3386fbc412b6', type: undefined }
                                      Feb 13 00:00:00 xoa xo-server[450]:   }
                                      Feb 13 00:00:00 xoa xo-server[450]: }
                                      Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.780Z xo:xo-mixins:backups-ng WARN no such object fe407fa9-96c6-0978-81ae-12b8874aa2dc {
                                      Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object fe407fa9-96c6-0978-81ae-12b8874aa2dc
                                      Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                      Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                      Feb 13 00:00:00 xoa xo-server[450]:     data: { id: 'fe407fa9-96c6-0978-81ae-12b8874aa2dc', type: undefined }
                                      Feb 13 00:00:00 xoa xo-server[450]:   }
                                      Feb 13 00:00:00 xoa xo-server[450]: }
                                      Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.781Z xo:xo-mixins:backups-ng WARN no such object 52fd8980-4a3f-9b29-a733-78abb9a603e5 {
                                      Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object 52fd8980-4a3f-9b29-a733-78abb9a603e5
                                      Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                      Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                      Feb 13 00:00:00 xoa xo-server[450]:     data: { id: '52fd8980-4a3f-9b29-a733-78abb9a603e5', type: undefined }
                                      Feb 13 00:00:00 xoa xo-server[450]:   }
                                      Feb 13 00:00:00 xoa xo-server[450]: }
                                      Feb 13 00:00:00 xoa xo-server[450]: 2023-02-13T00:00:00.785Z xo:xo-mixins:backups-ng WARN no such object d8e7f553-090d-2b67-acc3-83da0b58b8db {
                                      Feb 13 00:00:00 xoa xo-server[450]:   error: XoError: no such object d8e7f553-090d-2b67-acc3-83da0b58b8db
                                      Feb 13 00:00:00 xoa xo-server[450]:       at noSuchObject (/usr/local/lib/node_modules/xo-server/node_modules/xo-common/src/api-errors.js:21:32)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getObject (file:///usr/local/lib/node_modules/xo-server/src/xo.mjs:80:13)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at default.getXenServerIdByObject (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/xen-servers.mjs:199:26)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Xo.getXenServerIdByObject (/usr/local/lib/node_modules/xo-server/node_modules/lodash/_createBind.js:23:15)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at forEach (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:184:36)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Array.forEach (<anonymous>)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at executor (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/backups-ng/index.mjs:191:17)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:263:30
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs._runJob (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:292:22)
                                      Feb 13 00:00:00 xoa xo-server[450]:       at Jobs.runJobSequence (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/jobs/index.mjs:332:7) {
                                      Feb 13 00:00:00 xoa xo-server[450]:     code: 1,
                                      Feb 13 00:00:00 xoa xo-server[450]:     data: { id: 'd8e7f553-090d-2b67-acc3-83da0b58b8db', type: undefined }
                                      Feb 13 00:00:00 xoa xo-server[450]:   }
                                      Feb 13 00:00:00 xoa xo-server[450]: }
                                      Feb 13 01:00:01 xoa xo-server[450]: [WARN] plugin transport-email: Error: No recipients defined
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._formatError (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:774:19)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._setEnvelope (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:985:34)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.send (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:605:14)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at sendMessage (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:227:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at /usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:288:21
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.<anonymous> (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:209:17)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at Object.onceWrapper (node:events:627:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.emit (node:events:513:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._actionEHLO (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:1303:14)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._processResponse (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:932:20)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at SMTPConnection._onData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:739:14)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.SMTPConnection._onSocketData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:189:44)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.emit (node:events:513:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at addChunk (node:internal/streams/readable:324:12)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at readableAddChunk (node:internal/streams/readable:297:9)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)
                                      Feb 13 01:00:01 xoa xo-server[450]:     at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17)
                                      Feb 13 01:00:01 xoa xo-server[450]: 2023-02-13T01:00:01.340Z xo:xo-server WARN possibly unhandled rejection {
                                      Feb 13 01:00:01 xoa xo-server[450]:   error: Error: No recipients defined
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._formatError (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:774:19)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._setEnvelope (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:985:34)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.send (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:605:14)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at sendMessage (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:227:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at /usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-transport/index.js:288:21
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.<anonymous> (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:209:17)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at Object.onceWrapper (node:events:627:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.emit (node:events:513:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._actionEHLO (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:1303:14)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._processResponse (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:932:20)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at SMTPConnection._onData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:739:14)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.SMTPConnection._onSocketData (/usr/local/lib/node_modules/xo-server-transport-email/node_modules/nodemailer/lib/smtp-connection/index.js:189:44)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.emit (node:events:513:28)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at addChunk (node:internal/streams/readable:324:12)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at readableAddChunk (node:internal/streams/readable:297:9)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)
                                      Feb 13 01:00:01 xoa xo-server[450]:       at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
                                      Feb 13 01:00:01 xoa xo-server[450]:     code: 'EENVELOPE',
                                      Feb 13 01:00:01 xoa xo-server[450]:     command: 'API'
                                      Feb 13 01:00:01 xoa xo-server[450]:   }
                                      Feb 13 01:00:01 xoa xo-server[450]: }
                                      

                                      Thanks
                                      Simon

                                      1 Reply Last reply Reply Quote 0
                                      • olivierlambertO Offline
                                        olivierlambert Vates 🪐 Co-Founder CEO
                                        last edited by olivierlambert

                                        @KLSsandman please use the correct markdown syntax when you copy/paste console output. I already edited 4 times in a row your posts for that. If you want an example, just edit your own "corrected" post to see how to do it.

                                        K 1 Reply Last reply Reply Quote 0
                                        • K Offline
                                          KLSsandman @olivierlambert
                                          last edited by

                                          @olivierlambert Sorry, noted

                                          1 Reply Last reply Reply Quote 0
                                          • K Offline
                                            KLSsandman @florent
                                            last edited by

                                            @florent Thanks for the help so far, just wondering if there is anything else you could suggest please?

                                            Many thanks

                                            florentF 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post