XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Backup failing with "Error: HTTP connection has timed out"

    Scheduled Pinned Locked Moved Xen Orchestra
    backups-reportsbackup
    6 Posts 4 Posters 1.9k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • lawrencesystemsL Offline
      lawrencesystems Ambassador
      last edited by

      I am running xo-server 5.84.1 xo-web 5.90.0 built from source and since the latest update I am sometimes getting a Error: HTTP connection has timed out. I never had this error come up prior to the upgrade and It's not consistent and running the same backup a few more times does not cause it to fail each time. Not sure if it might be the system getting a bit overloaded when running a while running a larger the backup, it's never occurred on small VM's or delta backups. The XO vm is running on a Xeon E5-2670 0 @ 2.60GHz with 16 cores assigned & 4GB of memory.

      {
        "data": {
          "mode": "full",
          "reportWhen": "failure"
        },
        "id": "1638444000006",
        "jobId": "d1abe537-771e-4b62-8c3c-05d08c6854e8",
        "jobName": "Graylog",
        "message": "backup",
        "scheduleId": "6ea649f1-a45f-4665-af52-71ccad7edefe",
        "start": 1638444000006,
        "status": "failure",
        "infos": [
          {
            "data": {
              "vms": [
                "62570b52-e56b-a79b-ca5f-88803038532f"
              ]
            },
            "message": "vms"
          }
        ],
        "tasks": [
          {
            "data": {
              "type": "VM",
              "id": "62570b52-e56b-a79b-ca5f-88803038532f"
            },
            "id": "1638444000875",
            "message": "backup VM",
            "start": 1638444000875,
            "status": "failure",
            "tasks": [
              {
                "id": "1638444015767",
                "message": "snapshot",
                "start": 1638444015767,
                "status": "success",
                "end": 1638444017382,
                "result": "ada208e5-cb2f-5e4f-f6d1-19c64e6c8026"
              },
              {
                "data": {
                  "id": "b0323f4d-1828-4ad1-b9bd-550f38ff6cfa",
                  "type": "remote",
                  "isFull": true
                },
                "id": "1638444017480",
                "message": "export",
                "start": 1638444017480,
                "status": "failure",
                "tasks": [
                  {
                    "id": "1638444017488",
                    "message": "transfer",
                    "start": 1638444017488,
                    "status": "failure",
                    "end": 1638450296391,
                    "result": {
                      "canceled": false,
                      "method": "GET",
                      "url": "https://192.168.3.92/export/?ref=OpaqueRef%3A0ca54512-7a8c-452b-af08-bb18c83a0138&use_compression=zstd&session_id=OpaqueRef%3A09c8ef17-0f39-464a-a2c2-e3d9c0a51499&task_id=OpaqueRef%3Ae16c33e8-8360-4848-8a29-20ae6027e103",
                      "timeout": true,
                      "message": "HTTP connection has timed out",
                      "name": "Error",
                      "stack": "Error: HTTP connection has timed out\n    at IncomingMessage.emitAbortedError (/etc/xo/xo-builds/xen-orchestra-202112010617/node_modules/http-request-plus/index.js:83:19)\n    at Object.onceWrapper (events.js:519:28)\n    at IncomingMessage.emit (events.js:400:28)\n    at IncomingMessage.patchedEmit (/etc/xo/xo-builds/xen-orchestra-202112010617/@xen-orchestra/log/configure.js:118:17)\n    at IncomingMessage.emit (domain.js:475:12)\n    at TLSSocket.socketCloseListener (_http_client.js:432:11)\n    at TLSSocket.emit (events.js:412:35)\n    at TLSSocket.patchedEmit (/etc/xo/xo-builds/xen-orchestra-202112010617/@xen-orchestra/log/configure.js:118:17)\n    at TLSSocket.emit (domain.js:475:12)\n    at net.js:686:12"
                    }
                  }
                ],
                "end": 1638450296392,
                "result": {
                  "canceled": false,
                  "method": "GET",
                  "url": "https://192.168.3.92/export/?ref=OpaqueRef%3A0ca54512-7a8c-452b-af08-bb18c83a0138&use_compression=zstd&session_id=OpaqueRef%3A09c8ef17-0f39-464a-a2c2-e3d9c0a51499&task_id=OpaqueRef%3Ae16c33e8-8360-4848-8a29-20ae6027e103",
                  "timeout": true,
                  "message": "HTTP connection has timed out",
                  "name": "Error",
                  "stack": "Error: HTTP connection has timed out\n    at IncomingMessage.emitAbortedError (/etc/xo/xo-builds/xen-orchestra-202112010617/node_modules/http-request-plus/index.js:83:19)\n    at Object.onceWrapper (events.js:519:28)\n    at IncomingMessage.emit (events.js:400:28)\n    at IncomingMessage.patchedEmit (/etc/xo/xo-builds/xen-orchestra-202112010617/@xen-orchestra/log/configure.js:118:17)\n    at IncomingMessage.emit (domain.js:475:12)\n    at TLSSocket.socketCloseListener (_http_client.js:432:11)\n    at TLSSocket.emit (events.js:412:35)\n    at TLSSocket.patchedEmit (/etc/xo/xo-builds/xen-orchestra-202112010617/@xen-orchestra/log/configure.js:118:17)\n    at TLSSocket.emit (domain.js:475:12)\n    at net.js:686:12"
                }
              }
            ],
            "end": 1638450311244,
            "result": {
              "canceled": false,
              "method": "GET",
              "url": "https://192.168.3.92/export/?ref=OpaqueRef%3A0ca54512-7a8c-452b-af08-bb18c83a0138&use_compression=zstd&session_id=OpaqueRef%3A09c8ef17-0f39-464a-a2c2-e3d9c0a51499&task_id=OpaqueRef%3Ae16c33e8-8360-4848-8a29-20ae6027e103",
              "timeout": true,
              "message": "HTTP connection has timed out",
              "name": "Error",
              "stack": "Error: HTTP connection has timed out\n    at IncomingMessage.emitAbortedError (/etc/xo/xo-builds/xen-orchestra-202112010617/node_modules/http-request-plus/index.js:83:19)\n    at Object.onceWrapper (events.js:519:28)\n    at IncomingMessage.emit (events.js:400:28)\n    at IncomingMessage.patchedEmit (/etc/xo/xo-builds/xen-orchestra-202112010617/@xen-orchestra/log/configure.js:118:17)\n    at IncomingMessage.emit (domain.js:475:12)\n    at TLSSocket.socketCloseListener (_http_client.js:432:11)\n    at TLSSocket.emit (events.js:412:35)\n    at TLSSocket.patchedEmit (/etc/xo/xo-builds/xen-orchestra-202112010617/@xen-orchestra/log/configure.js:118:17)\n    at TLSSocket.emit (domain.js:475:12)\n    at net.js:686:12"
            }
          }
        ],
        "end": 1638450311245
      }
      
      1 Reply Last reply Reply Quote 0
      • lawrencesystemsL Offline
        lawrencesystems Ambassador
        last edited by

        Just noticed there was another update that makes a few changes to backup, I did not see the same error message that I am getting but I updated to the latest version and I will do some more testing to see if the error message comes back.
        https://github.com/vatesfr/xen-orchestra/blob/master/CHANGELOG.md

        1 Reply Last reply Reply Quote 0
        • DanpD Offline
          Danp Pro Support Team
          last edited by

          Was just about to post suggesting that you update again. Hopefully the latest fixes address your issue.

          1 Reply Last reply Reply Quote 1
          • olivierlambertO Online
            olivierlambert Vates ๐Ÿช Co-Founder CEO
            last edited by

            Also HTTP connection has timed out might be a real network issue, but first let's rule out other possibilities by using the latest commit on master ๐Ÿ™‚

            I 1 Reply Last reply Reply Quote 0
            • I Offline
              iLix @olivierlambert
              last edited by

              I am also seeing this on a large vm(almost 2tb), the sub 200GB VMโ€™s backup fine. I canโ€™t see any errors on the switches. I will add more details about my setup tomorrow.

              1 Reply Last reply Reply Quote 0
              • lawrencesystemsL Offline
                lawrencesystems Ambassador
                last edited by

                Since the update I have run full backup that have taken over an hour to complete due to their size and many smaller delta backups and the error has not come up again. I assume whatever was causing the error was fixed in the xo-server 5.84.2 update.

                1 Reply Last reply Reply Quote 1
                • northportioN northportio referenced this topic on
                • First post
                  Last post