XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    backup failed

    Scheduled Pinned Locked Moved Xen Orchestra
    35 Posts 7 Posters 14.5k Views 1 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • olivierlambertO Offline
      olivierlambert Vates πŸͺ Co-Founder CEO
      last edited by

      It's indeed the last one. Are you sure it's properly rebuilt?

      Anyway, that might be something else. @julien-f any idea?

      B DanpD 2 Replies Last reply Reply Quote 0
      • B Offline
        Black_Sam @olivierlambert
        last edited by

        @olivierlambert

        I'm sure I did a complete rebuild.

        1 Reply Last reply Reply Quote 0
        • DanpD Offline
          Danp Pro Support Team @olivierlambert
          last edited by

          @olivierlambert @julien-f

          FWIW, I have also started having this same issue after updating to the latest sources.

          Here are some details --

          Commit d44509b2cd394e3a38dc4ba392cc54dd2f50e89f  Working backing
          Commit 56e4847b6bb85da8ae2dc09e8e9fb7a0db36070a Missing writer issue
          Commit 9a8138d07bc1a5f457ebcb4bbff83cd07cda80ed all targets have failed, step: writer.beforeBackup()
          
          julien-fJ 1 Reply Last reply Reply Quote 0
          • julien-fJ Offline
            julien-f Vates πŸͺ Co-Founder XO Team @Danp
            last edited by

            Thank you all, I'm investigating.

            julien-fJ 1 Reply Last reply Reply Quote 0
            • julien-fJ Offline
              julien-f Vates πŸͺ Co-Founder XO Team @julien-f
              last edited by

              Should be fixed, sorry for this 😬

              Thanks all for your feedback!

              B JoeArnonJ 3 Replies Last reply Reply Quote 2
              • B Offline
                Black_Sam @julien-f
                last edited by

                @julien-f

                a full backup of my VMs is in progress ... if any issues arise i will post it here. Thank you for your commitment! That is awesome!

                1 Reply Last reply Reply Quote 1
                • JoeArnonJ Offline
                  JoeArnon @julien-f
                  last edited by

                  @julien-f
                  Thanks a lot julien

                  1 Reply Last reply Reply Quote 1
                  • B Offline
                    Black_Sam @julien-f
                    last edited by Black_Sam

                    @julien-f

                    Full backup are finished. 3 of 4 VMs are backuped. The last one failed.

                    the backup of my nextcloud server failed with following error:

                    {
                      "data": {
                        "mode": "full",
                        "reportWhen": "failure"
                      },
                      "id": "1623934497160",
                      "jobId": "f67f6ed7-f013-4ff5-812e-520af137dc37",
                      "jobName": "Full Backup",
                      "message": "backup",
                      "scheduleId": "e393ec6f-e3f6-4e8f-a6aa-49de1adc02ac",
                      "start": 1623934497160,
                      "status": "failure",
                      "infos": [
                        {
                          "data": {
                            "vms": [
                              "03500917-c7e5-0bd4-e684-ec3ffa33a455",
                              "43e2f8f3-b11c-b6c3-a405-c810aecf42c7",
                              "750e2ee5-5d3d-d57c-2d82-f1fbfa2caa95",
                              "95d56ff8-9262-8281-91d3-261795c4d75c"
                            ]
                          },
                          "message": "vms"
                        }
                      ],
                      "tasks": [
                        {
                          "data": {
                            "type": "VM",
                            "id": "03500917-c7e5-0bd4-e684-ec3ffa33a455"
                          },
                          "id": "1623934501609",
                          "message": "backup VM",
                          "start": 1623934501609,
                          "status": "failure",
                          "tasks": [
                            {
                              "id": "1623934502218",
                              "message": "snapshot",
                              "start": 1623934502218,
                              "status": "success",
                              "end": 1623934507910,
                              "result": "87149237-162e-3e72-7f86-7e3463a18cdf"
                            },
                            {
                              "data": {
                                "id": "e916984e-b326-4c2a-a8b1-d94c28a22953",
                                "type": "remote",
                                "isFull": true
                              },
                              "id": "1623934507940",
                              "message": "export",
                              "start": 1623934507940,
                              "status": "failure",
                              "tasks": [
                                {
                                  "id": "1623934507952",
                                  "message": "transfer",
                                  "start": 1623934507952,
                                  "status": "failure",
                                  "end": 1623935709862,
                                  "result": {
                                    "canceled": true,
                                    "method": "GET",
                                    "url": "https://192.168.32.2/export/?ref=OpaqueRef%3Aeaeffed6-2256-48fd-aa29-12808bc5ef82&use_compression=false&session_id=OpaqueRef%3Ac12804b3-36be-49c3-9578-051055335fbc&task_id=OpaqueRef%3A2ee9dfc0-7776-4ea2-adb6-3d86eaa4fc88",
                                    "message": "HTTP request has been canceled",
                                    "name": "Error",
                                    "stack": "Error: HTTP request has been canceled\n    at IncomingMessage.emitAbortedError (/opt/xen-orchestra/node_modules/http-request-plus/index.js:79:19)\n    at Object.onceWrapper (events.js:481:28)\n    at IncomingMessage.emit (events.js:375:28)\n    at IncomingMessage.patchedEmit (/opt/xen-orchestra/@xen-orchestra/log/configure.js:93:17)\n    at IncomingMessage.emit (domain.js:470:12)\n    at TLSSocket.socketCloseListener (_http_client.js:432:11)\n    at TLSSocket.emit (events.js:387:35)\n    at TLSSocket.patchedEmit (/opt/xen-orchestra/@xen-orchestra/log/configure.js:93:17)\n    at TLSSocket.emit (domain.js:470:12)\n    at net.js:675:12"
                                  }
                                }
                              ],
                              "end": 1623935709863,
                              "result": {
                                "canceled": true,
                                "method": "GET",
                                "url": "https://192.168.32.2/export/?ref=OpaqueRef%3Aeaeffed6-2256-48fd-aa29-12808bc5ef82&use_compression=false&session_id=OpaqueRef%3Ac12804b3-36be-49c3-9578-051055335fbc&task_id=OpaqueRef%3A2ee9dfc0-7776-4ea2-adb6-3d86eaa4fc88",
                                "message": "HTTP request has been canceled",
                                "name": "Error",
                                "stack": "Error: HTTP request has been canceled\n    at IncomingMessage.emitAbortedError (/opt/xen-orchestra/node_modules/http-request-plus/index.js:79:19)\n    at Object.onceWrapper (events.js:481:28)\n    at IncomingMessage.emit (events.js:375:28)\n    at IncomingMessage.patchedEmit (/opt/xen-orchestra/@xen-orchestra/log/configure.js:93:17)\n    at IncomingMessage.emit (domain.js:470:12)\n    at TLSSocket.socketCloseListener (_http_client.js:432:11)\n    at TLSSocket.emit (events.js:387:35)\n    at TLSSocket.patchedEmit (/opt/xen-orchestra/@xen-orchestra/log/configure.js:93:17)\n    at TLSSocket.emit (domain.js:470:12)\n    at net.js:675:12"
                              }
                            }
                          ],
                          "end": 1623936374410,
                          "result": {
                            "message": "this.delete is not a function",
                            "name": "TypeError",
                            "stack": "TypeError: this.delete is not a function\n    at VmBackup._callWriters (/opt/xen-orchestra/@xen-orchestra/backups/_VmBackup.js:115:20)\n    at async VmBackup._copyFull (/opt/xen-orchestra/@xen-orchestra/backups/_VmBackup.js:248:5)\n    at async VmBackup.run (/opt/xen-orchestra/@xen-orchestra/backups/_VmBackup.js:383:9)"
                          }
                        },
                        {
                          "data": {
                            "type": "VM",
                            "id": "43e2f8f3-b11c-b6c3-a405-c810aecf42c7"
                          },
                          "id": "1623934501635",
                          "message": "backup VM",
                          "start": 1623934501635,
                          "status": "success",
                          "tasks": [
                            {
                              "id": "1623934502223",
                              "message": "snapshot",
                              "start": 1623934502223,
                              "status": "success",
                              "end": 1623934511607,
                              "result": "63b42581-fc41-2f6f-a65a-4e6ce2d1f751"
                            },
                            {
                              "data": {
                                "id": "e916984e-b326-4c2a-a8b1-d94c28a22953",
                                "type": "remote",
                                "isFull": true
                              },
                              "id": "1623934511673",
                              "message": "export",
                              "start": 1623934511673,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1623934511686",
                                  "message": "transfer",
                                  "start": 1623934511686,
                                  "status": "success",
                                  "end": 1623934939556,
                                  "result": {
                                    "size": 12547841024
                                  }
                                }
                              ],
                              "end": 1623934939577
                            }
                          ],
                          "end": 1623934940295
                        },
                        {
                          "data": {
                            "type": "VM",
                            "id": "750e2ee5-5d3d-d57c-2d82-f1fbfa2caa95"
                          },
                          "id": "1623934940296",
                          "message": "backup VM",
                          "start": 1623934940296,
                          "status": "success",
                          "tasks": [
                            {
                              "id": "1623934940332",
                              "message": "snapshot",
                              "start": 1623934940332,
                              "status": "success",
                              "end": 1623934942195,
                              "result": "6ea682be-d8ee-d7af-e4db-26749bbd1d9b"
                            },
                            {
                              "data": {
                                "id": "e916984e-b326-4c2a-a8b1-d94c28a22953",
                                "type": "remote",
                                "isFull": true
                              },
                              "id": "1623934942234",
                              "message": "export",
                              "start": 1623934942234,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1623934942295",
                                  "message": "transfer",
                                  "start": 1623934942295,
                                  "status": "success",
                                  "end": 1623935204057,
                                  "result": {
                                    "size": 8491254272
                                  }
                                }
                              ],
                              "end": 1623935204069
                            }
                          ],
                          "end": 1623935204770
                        },
                        {
                          "data": {
                            "type": "VM",
                            "id": "95d56ff8-9262-8281-91d3-261795c4d75c"
                          },
                          "id": "1623935204771",
                          "message": "backup VM",
                          "start": 1623935204771,
                          "status": "success",
                          "tasks": [
                            {
                              "id": "1623935204826",
                              "message": "snapshot",
                              "start": 1623935204826,
                              "status": "success",
                              "end": 1623935206750,
                              "result": "59f973c4-432c-7575-4934-1f868ecbb736"
                            },
                            {
                              "data": {
                                "id": "e916984e-b326-4c2a-a8b1-d94c28a22953",
                                "type": "remote",
                                "isFull": true
                              },
                              "id": "1623935206781",
                              "message": "export",
                              "start": 1623935206781,
                              "status": "success",
                              "tasks": [
                                {
                                  "id": "1623935206851",
                                  "message": "transfer",
                                  "start": 1623935206851,
                                  "status": "success",
                                  "end": 1623935536915,
                                  "result": {
                                    "size": 11896770560
                                  }
                                }
                              ],
                              "end": 1623935536933
                            }
                          ],
                          "end": 1623935537657
                        }
                      ],
                      "end": 1623936374413
                    }
                    

                    The output of "journalctl -u xo-server -f -n 50" can you see below:

                    Jun 17 15:15:09 xoce xo-server[14386]: 2021-06-17T13:15:09.679Z xo:backups:worker WARN possibly unhandled rejection {
                    Jun 17 15:15:09 xoce xo-server[14386]:   error: Error: HTTP request has been canceled
                    Jun 17 15:15:09 xoce xo-server[14386]:       at IncomingMessage.emitAbortedError (/opt/xen-orchestra/node_modules/http-request-plus/index.js:79:19)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at Object.onceWrapper (events.js:481:28)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at IncomingMessage.emit (events.js:375:28)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at IncomingMessage.patchedEmit (/opt/xen-orchestra/@xen-orchestra/log/configure.js:93:17)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at IncomingMessage.emit (domain.js:470:12)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at TLSSocket.socketCloseListener (_http_client.js:432:11)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at TLSSocket.emit (events.js:387:35)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at TLSSocket.patchedEmit (/opt/xen-orchestra/@xen-orchestra/log/configure.js:93:17)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at TLSSocket.emit (domain.js:470:12)
                    Jun 17 15:15:09 xoce xo-server[14386]:       at net.js:675:12 {
                    Jun 17 15:15:09 xoce xo-server[14386]:     canceled: true,
                    Jun 17 15:15:09 xoce xo-server[14386]:     method: 'GET',
                    Jun 17 15:15:09 xoce xo-server[14386]:     url: 'https://192.168.32.2/export/?ref=OpaqueRef%3Aeaeffed6-2256-48fd-aa29-12808bc5ef82&use_compression=false&session_id=OpaqueRef%3Ac12804b3-36be-49c3-9578-051055335fbc&task_id=OpaqueRef%3A2ee9dfc0-7776-4ea2-adb6-3d86eaa4fc88'
                    Jun 17 15:15:09 xoce xo-server[14386]:   }
                    Jun 17 15:15:09 xoce xo-server[14386]: }
                    

                    The backup breaks off at approx. 44-45% with the error mentioned above ...

                    If that should be a new thread just give a decision then I'll open a new one.

                    julien-fJ 1 Reply Last reply Reply Quote 0
                    • julien-fJ Offline
                      julien-f Vates πŸͺ Co-Founder XO Team @Black_Sam
                      last edited by

                      @black_sam Please makes sure that you are using the latest xen-api lib. Also, this might be a XCP-ng/XenServer issues, please check your host's logs. Out of curiosity, are you using compression in your backup job?

                      B 1 Reply Last reply Reply Quote 0
                      • B Offline
                        Black_Sam @julien-f
                        last edited by Black_Sam

                        @julien-f

                        xcp-ng 8.2.0 is fully patched so ist should be up to date.
                        I didn't setup any compression options. I only configurated the backup jobs over xen orchestra

                        €dit:
                        hmmm... I think it was a problem with my network... after shifting the host into an other subnet without VLAN the backup finshed without any issue.

                        Thank you for your support

                        julien-fJ 1 Reply Last reply Reply Quote 0
                        • julien-fJ Offline
                          julien-f Vates πŸͺ Co-Founder XO Team @Black_Sam
                          last edited by

                          @black_sam Thanks for your feedback πŸ™‚

                          1 Reply Last reply Reply Quote 1
                          • B berish.lohith referenced this topic on

                          Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                          Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                          With your input, this post could be even better πŸ’—

                          Register Login
                          • First post
                            Last post