XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup jobs started to fail after last XOA update

    Scheduled Pinned Locked Moved Xen Orchestra
    13 Posts 4 Posters 1.2k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ForzaF Offline
      Forza
      last edited by

      Since the latest XOA update last week I am getting failed backup jobs. They look like this:

      •	UUID: 5ee88933-d28a-f4ca-7dde-7490f3564398
      •	Start time: Saturday, December 3rd 2022, 4:31:43 pm
      •	End time: Sunday, December 4th 2022, 4:33:09 pm
      •	Duration: a day
      •	Error: self-signed certificate
      

      It seems I had to enable "Unauthorized Certificates" in the servers settings to make backups work again. Is this a new setting or is this just a coincidence somehow?
      6a841bf3-4e03-48ff-8ba1-34b7ef851261-image.png a7405556-0687-4334-8a71-13791a432bca-image.png

      Current version: 5.77.0 - XOA build: 20210102

      - node: 18.12.1
      - npm: 8.19.2
      - xen-orchestra-upload-ova: 0.1.4
      - xo-server: 5.107.1
      - xo-server-auth-github-enterprise: 0.2.2
      - xo-server-auth-google-enterprise: 0.2.2
      - xo-server-auth-ldap-enterprise: 0.10.6
      - xo-server-auth-saml-enterprise: 0.10.1
      - xo-server-backup-reports-enterprise: 0.17.2
      - xo-server-netbox-enterprise: 0.3.5
      - xo-server-telemetry: 0.5.0
      - xo-server-transport-email-enterprise: 0.6.0
      - xo-server-transport-icinga2-enterprise: 0.1.1
      - xo-server-transport-nagios-enterprise: 1.0.0
      - xo-server-transport-slack-enterprise: 0.0.0
      - xo-server-transport-xmpp-enterprise: 0.1.1
      - xo-server-usage-report-enterprise: 0.10.2
      - xo-server-xoa: 0.17.0
      - xo-web-enterprise: 5.108.0
      - xoa-cli: 0.33.0
      - xoa-updater: 0.45.0
      

      Looking at the error log I see that XOA is trying HTTPS instead of HTTP when communicating with the XCP-ng host:

      "result": {
              "code": "DEPTH_ZERO_SELF_SIGNED_CERT",
              "url": "https://10.254.9.2/export/?ref=OpaqueRef:3377f666-5fdc-41e5-aebf-39c7ffa5ad3f&use_compression=zstd&session_id=OpaqueRef:dd989ec7-60ac-45af-8fcd-faf542f8e62a&task_id=OpaqueRef:5617043e-af92-4023-9482-043094180083",
      ...
              "VM": "OpaqueRef:3377f666-5fdc-41e5-aebf-39c7ffa5ad3f",
              "message": "self-signed certificate",
              "name": "Error",
              "stack": "Error: self-signed certificate\n    at TLSSocket.onConnectSecure (node:_tls_wrap:1538:34)\n    at TLSSocket.emit (node:events:513:28)\n    at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:135:17)\n    at TLSSocket._finishInit (node:_tls_wrap:952:8)\n    at ssl.onhandshakedone (node:_tls_wrap:733:12)\n    at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17)"
            }
      
      1 Reply Last reply Reply Quote 0
      • olivierlambertO Online
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by olivierlambert

        Hi,

        Are you using XO from the sources or XOA?

        edit: it seems to be XOA, missed that later in your post 😄

        Does it ring bell julien-f ?

        ForzaF 1 Reply Last reply Reply Quote 0
        • ForzaF Offline
          Forza @olivierlambert
          last edited by

          olivierlambert said in Backup jobs started to fail after last XOA update:

          Indeed, I use XOA with Enterprise license 🙂

          julien-fJ 1 Reply Last reply Reply Quote 0
          • olivierlambertO Online
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            In this case, it's better to create support tickets than use the forum, this way your issue can be treated in priority 🙂

            1 Reply Last reply Reply Quote 0
            • julien-fJ Offline
              julien-f Vates 🪐 Co-Founder XO Team @Forza
              last edited by

              Forza This is perfectly normal, XO is using HTTPS to connect to hosts by default for many years, and if you are not using an HTTPS certificate trusted by common authorities, you need to enable this setting.

              ForzaF 1 Reply Last reply Reply Quote 0
              • ForzaF Offline
                Forza @julien-f
                last edited by

                julien-f said in Backup jobs started to fail after last XOA update:

                Forza This is perfectly normal, XO is using HTTPS to connect to hosts by default for many years, and if you are not using an HTTPS certificate trusted by common authorities, you need to enable this setting.

                I know https:// is default, but I remeber we could enable http:// to improve backup and migration performance. Is not possible any more?

                julien-fJ S 2 Replies Last reply Reply Quote 0
                • julien-fJ Offline
                  julien-f Vates 🪐 Co-Founder XO Team @Forza
                  last edited by

                  Forza Yes, it's possible, simply prefix your host address by http:// in Settings/Servers.

                  If that does not work, open a support ticket and a tunnel and I'll investigate 🙂

                  ForzaF 1 Reply Last reply Reply Quote 0
                  • ForzaF Offline
                    Forza @julien-f
                    last edited by

                    julien-f said in Backup jobs started to fail after last XOA update:

                    Forza Yes, it's possible, simply prefix your host address by http:// in Settings/Servers.

                    If that does not work, open a support ticket and a tunnel and I'll investigate 🙂

                    This is why I was surprised. I have been using http:// for a long time now. But since I changed the "Unauthorized Certificates" it started to work again.

                    a1a1da48-2962-4324-af64-3c72e01846bf-image.png

                    If you think it is important, I can open a tunnel for you to look?

                    julien-fJ 1 Reply Last reply Reply Quote 0
                    • S Offline
                      sluflyer06 @Forza
                      last edited by

                      Forza Wow, I just tried that out and backup speed increased by 48%.

                      1 Reply Last reply Reply Quote 1
                      • julien-fJ Offline
                        julien-f Vates 🪐 Co-Founder XO Team @Forza
                        last edited by

                        Forza Yes please 🙂

                        sluflyer06 We are currently investigating this, whether the perf issue comes from XO or XCP-ng.

                        julien-fJ 1 Reply Last reply Reply Quote 1
                        • julien-fJ Offline
                          julien-f Vates 🪐 Co-Founder XO Team @julien-f
                          last edited by

                          For anyone interested:

                          Okay, after investigation, it appears that when the pool master is an HTTP request for exporting a VDI on another host (for instance because the VDI is on a local SR), it explicitly specify HTTPS in the new URL.

                          I'm not sure it's a good idea to ignore this as I'm afraid it may cause issues in the future.

                          Anyway XO behavior has not changed recently as it comes from the host.

                          We are currently investing the perf impact of HTTPS and hopefully we'll find a way to improve it.

                          ForzaF 1 Reply Last reply Reply Quote 0
                          • ForzaF Offline
                            Forza @julien-f
                            last edited by

                            julien-f Would it help if I add the pool member to the "Servers" list as http:// ?
                            fc20be5f-58b1-4320-a0c9-fd21410988ae-image.png

                            julien-fJ 1 Reply Last reply Reply Quote 0
                            • julien-fJ Offline
                              julien-f Vates 🪐 Co-Founder XO Team @Forza
                              last edited by

                              Forza No, it's not possible to add multiple hosts of the same pool to XO.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post