Backup NG: "value" argument is out of bounds

asai

Greetings,

I got an alert this morning that a Backup NG scheduled backup on a VM had failed with the error: "value" argument is out of bounds.

I am unable to find anything on-line related to this error. Can anyone shed light on this or where to start looking for answers? Thank you.

Asai

olivierlambert

Hi there,

We need more info:

Are you using XOA or XO from the sources?
What's your current version?

asai

Thank you, @olivierlambert

I am using this github script to install XOA https://github.com/Jarli01/xenorchestra_installer

And my XO version is: 5.44.0

olivierlambert

You don't really have an XO version using it from the source, you have a version for xo-web, xo-server etc.

So can you report first your issue on the Github repo of people who made the script?

In the meantime I'm asking our lead dev: @julien-f does error: "value" argument is out of bounds ring any bell?

asai

@olivierlambert , I should add that these backups have been running for weeks, 3 times a week, with no problems, this is the first time we've seen any indication that there's an issue.

julien-f

@olivierlambert It looks like a bug in our VHD implementation, but it's not precise enough for me to narrow it down.

@asai Do you have any other info?

asai

@julien-f , what do you need from me to provide more info? Which logs can I get that info from?

Pending your response, I will start poking around the logs myself, and see if I can turn up anything useful.

asai

@julien-f , if it's helpful, I recently increased the size of the VMs disk by 40 GB.

asai

@julien-f , here is the error message from the Backup NG report

Snapshot 
Start: Sep 11, 2019, 01:00:00 AM
End: Sep 11, 2019, 01:00:06 AM
 XenBackupsUnify 
 transfer 
Start: Sep 11, 2019, 01:00:07 AM
End: Sep 11, 2019, 01:07:53 AM
Duration: 8 minutes
Size: 2.38 GiB
Speed: 5.22 MiB/s
 merge 
Start: Sep 11, 2019, 01:07:53 AM
End: Sep 11, 2019, 01:11:03 AM
Duration: 3 minutes
Error: "value" argument is out of bounds
Start: Sep 11, 2019, 01:00:07 AM
End: Sep 11, 2019, 01:11:03 AM
Duration: 11 minutes
Error: "value" argument is out of bounds

julien-f

@asai said in Backup NG: "value" argument is out of bounds:

if it's helpful, I recently increased the size of the VMs disk by 40 GB.

Yep, thank you that may be relevant.

Also, you can copy in the clipboard the full backup log from the details modal of the run.

asai

@julien-f

Thanks for looking at this:

2019-09-11T08_00_00.003Z - backup NG.txt

hecki

Same problem here with current stable xo-server 5.53.0, xo-web 5.53.3 (build from source).
I also increased the virtual disk of the VM a week before.

@asai, @julien-f are ther any news regarding this issue?

hecki

Here is the failed backup log:

    {
      "data": {
        "type": "VM",
        "id": "e93c337e-3532-c5fe-6442-25587d65c565"
      },
      "id": "1578874920072",
      "message": "Starting backup of Calypso (8341bb16-5e0f-42b8-ac81-a68ac532d42b)",
      "start": 1578874920072,
      "status": "failure",
      "tasks": [
        {
          "id": "1578874920072:0",
          "message": "snapshot",
          "start": 1578874920072,
          "status": "success",
          "end": 1578874931254,
          "result": "ab38fd4d-6d97-d630-f8d8-808b878a9c17"
        },
        {
          "id": "1578874931274",
          "message": "add metadata to snapshot",
          "start": 1578874931274,
          "status": "success",
          "end": 1578874931320
        },
        {
          "id": "1578874931449",
          "message": "waiting for uptodate snapshot record",
          "start": 1578874931449,
          "status": "success",
          "end": 1578874931676
        },
        {
          "id": "1578874932280",
          "message": "start snapshot export",
          "start": 1578874932280,
          "status": "success",
          "end": 1578874932280
        },
        {
          "data": {
            "id": "68c50187-ec00-45b8-9ada-fda127cf1d9c",
            "isFull": false,
            "type": "remote"
          },
          "id": "1578874932280:1",
          "message": "export",
          "start": 1578874932280,
          "status": "failure",
          "tasks": [
            {
              "id": "1578874932377",
              "message": "transfer",
              "start": 1578874932377,
              "status": "success",
              "end": 1578879743091,
              "result": {
                "size": 19443764736
              }
            },
            {
              "id": "1578879743101",
              "message": "merge",
              "start": 1578879743101,
              "status": "failure",
              "end": 1578882356886,
              "result": {
                "type": "TypeError",
                "message": "\"value\" argument is out of bounds",
                "name": "TypeError",
                "stack": "TypeError: \"value\" argument is out of bounds\n    at checkInt (buffer.js:1185:11)\n    at Buffer.writeUInt32BE (buffer.js:1278:5)\n    at Vhd._setBatEntry (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:236:16)\n    at Vhd._createBlock (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:243:103)\n    at Vhd._writeEntireBlock (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:265:30)\n    at Vhd.coalesceBlock (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:319:20)\n    at <anonymous>"
              }
            }
          ],
          "end": 1578882356886,
          "result": {
            "type": "TypeError",
            "message": "\"value\" argument is out of bounds",
            "name": "TypeError",
            "stack": "TypeError: \"value\" argument is out of bounds\n    at checkInt (buffer.js:1185:11)\n    at Buffer.writeUInt32BE (buffer.js:1278:5)\n    at Vhd._setBatEntry (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:236:16)\n    at Vhd._createBlock (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:243:103)\n    at Vhd._writeEntireBlock (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:265:30)\n    at Vhd.coalesceBlock (/opt/xen-orchestra/packages/vhd-lib/dist/vhd.js:319:20)\n    at <anonymous>"
          }
        },
        {
          "id": "1578882356898",
          "message": "set snapshot.other_config[xo:backup:exported]",
          "start": 1578882356898,
          "status": "success",
          "end": 1578882356907
        }
      ],
      "end": 1578882361453
    },

ronivay

Stumbled to the same error. Retry finished okay and no issues really. Couldn't really find anything except this thread and no info if this is something that will be fixed/is fixed already.

Not running the latest versions (xo-server 5.50.1, xo-web 5.50.2), but because of the above, i'm sharing logs as well if found useful.

    {
      "data": {
        "type": "VM",
        "id": "ae0fd9f4-6915-475e-7809-c3c501177c51"
      },
      "id": "1585875600008:4",
      "message": "Starting backup of vmname (8b5016a9-6ffa-4614-8e6f-c87fe4bcb93f)",
      "start": 1585875600008,
      "status": "failure",
      "tasks": [
        {
          "id": "1585875600009",
          "message": "snapshot",
          "start": 1585875600009,
          "status": "success",
          "end": 1585875651531,
          "result": "49545d5f-633c-3bb2-ca7e-8d1cfdb60fc9"
        },
        {
          "id": "1585875651534",
          "message": "add metadata to snapshot",
          "start": 1585875651534,
          "status": "success",
          "end": 1585875651595
        },
        {
          "id": "1585875651700",
          "message": "waiting for uptodate snapshot record",
          "start": 1585875651700,
          "status": "success",
          "end": 1585875651962
        },
        {
          "id": "1585875652432",
          "message": "start snapshot export",
          "start": 1585875652432,
          "status": "success",
          "end": 1585875652432
        },
        {
          "data": {
            "id": "aac606cd-a5f0-4a8e-935c-854f144145b7",
            "isFull": false,
            "type": "remote"
          },
          "id": "1585875652432:1",
          "message": "export",
          "start": 1585875652432,
          "status": "failure",
          "tasks": [
            {
              "id": "1585875652633",
              "message": "transfer",
              "start": 1585875652633,
              "status": "success",
              "end": 1585875804106,
              "result": {
                "size": 444748288
              }
            },
            {
              "id": "1585875804106:0",
              "message": "merge",
              "start": 1585875804106,
              "status": "failure",
              "end": 1585875830977,
              "result": {
                "type": "TypeError",
                "message": "\"value\" argument is out of bounds",
                "name": "TypeError",
                "stack": "TypeError: \"value\" argument is out of bounds\n    at checkInt (buffer.js:1185:11)\n    at Buffer.writeUInt32BE (buffer.js:1278:5)\n    at Vhd._setBatEntry (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:244:16)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:254:111\n    at Generator.next (<anonymous>)\n    at asyncGeneratorStep (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:24:103)\n    at _next (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:26:194)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:26:364\n    at Promise._execute (/etc/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)\n    at Promise._resolveFromExecutor (/etc/xen-orchestra/node_modules/bluebird/js/release/promise.js:488:18)\n    at new Promise (/etc/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:26:97\n    at Vhd.createBlock (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:256:7)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:282:34\n    at Generator.next (<anonymous>)\n    at asyncGeneratorStep (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:24:103)"
              }
            }
          ],
          "end": 1585875830977,
          "result": {
            "type": "TypeError",
            "message": "\"value\" argument is out of bounds",
            "name": "TypeError",
            "stack": "TypeError: \"value\" argument is out of bounds\n    at checkInt (buffer.js:1185:11)\n    at Buffer.writeUInt32BE (buffer.js:1278:5)\n    at Vhd._setBatEntry (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:244:16)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:254:111\n    at Generator.next (<anonymous>)\n    at asyncGeneratorStep (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:24:103)\n    at _next (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:26:194)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:26:364\n    at Promise._execute (/etc/xen-orchestra/node_modules/bluebird/js/release/debuggability.js:313:9)\n    at Promise._resolveFromExecutor (/etc/xen-orchestra/node_modules/bluebird/js/release/promise.js:488:18)\n    at new Promise (/etc/xen-orchestra/node_modules/bluebird/js/release/promise.js:79:10)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:26:97\n    at Vhd.createBlock (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:256:7)\n    at /etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:282:34\n    at Generator.next (<anonymous>)\n    at asyncGeneratorStep (/etc/xen-orchestra/packages/vhd-lib/dist/vhd.js:24:103)"
          }
        },
        {
          "id": "1585875830979",
          "message": "set snapshot.other_config[xo:backup:exported]",
          "start": 1585875830979,
          "status": "success",
          "end": 1585875831050
        }
      ],
      "end": 1585875918847
    },

olivierlambert

Hi,

Please check if you have the same issue with XOA on latest. Also retry with latest XO from source on master branch. We don't do any kind of support of a source install that's not a latest GitHub commit it would be impossible to maintain that!