XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    VDI_IO_ERROR(Device I/O errors) when you run scheduled backup

    Scheduled Pinned Locked Moved Xen Orchestra
    66 Posts 11 Posters 17.6k Views 7 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S Offline
      shorian @tuxen
      last edited by

      @tuxen Target has plenty of space, over 5Tb free or about 20x the cumulative VM sizes. No NFS involved, it’s a locally mounted ext4 raid 1 array on the target box.

      If same backup takes place behind the firewall it runs successfully 95% of the time, across the WAN it fails 95% of the time. Both over a 1gbps link.

      Sometimes the failures clean themselves up, sometimes end up with a VM/disk marked [importing.....<backup name><VM name>] that need to be manually removed.

      Any help hugely appreciated.

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates πŸͺ Co-Founder CEO
        last edited by

        Interesting. It's like the data stream is interrupted somehow for a bit and that's enough to trigger the issue.

        1 Reply Last reply Reply Quote 1
        • EddieCh08666741E Offline
          EddieCh08666741
          last edited by

          i have similar issue like this too. I'm using ext4 and it was perfectly fine when i'm using 7.6. After upgrading to ext4 and 8.2 fresh install. The CR dont work anymore.

          1 Reply Last reply Reply Quote 0
          • EddieCh08666741E Offline
            EddieCh08666741
            last edited by EddieCh08666741

            I just tried installing Xen Orchestra from the sources on Debian 11. The same CR works well.

            My previous Xen Orchestra from the sources ubuntu 18 having issue with VDI ERROR. Will do more testing.

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates πŸͺ Co-Founder CEO
              last edited by

              Note: XOA is only the version distributed by Vates. Everything else is "Xen Orchestra from the sources" πŸ˜‰

              EddieCh08666741E 1 Reply Last reply Reply Quote 0
              • EddieCh08666741E Offline
                EddieCh08666741 @olivierlambert
                last edited by

                @olivierlambert Thanks Corrected. I'm so happy for XCP-NG. I'm one of the early backers in Kickstarter. Hang the shirt up in our office.

                1 Reply Last reply Reply Quote 1
                • R Offline
                  rauly94
                  last edited by Danp

                  Good Morning guys,

                  it looks like I'm having the same issue on a host that I'm doing replications. On the normal Delta Backups I'm not getting this error. But I'm getting it on the schedule Replication for only 1 of the vm's. Here is what comes on the log for that VM.

                  "id": "1679486747187",
                                "message": "transfer",
                                "start": 1679486747187,
                                "status": "failure",
                                "end": 1679487904889,
                                "result": {
                                  "code": "VDI_IO_ERROR",
                                  "params": [
                                    "Device I/O errors"
                                  ],
                                  "url": "https://192.168.2.11/import_raw_vdi/?format=vhd&vdi=OpaqueRef%3Aa5f60c35-64c2-497e-ae15-77aa63d14274&session_id=OpaqueRef%3A1593dbd0-4347-4dee-ad02-58ed84f6dbf6&task_id=OpaqueRef%3Aced050e9-1f49-4bbe-8b74-ef787d536d62",
                                  "task": {
                                    "uuid": "a7e3ba73-9d02-c211-48be-6a246828211c",
                                    "name_label": "[XO] Importing content into VDI HEALmycroft 0",
                                    "name_description": "",
                                    "allowed_operations": [],
                                    "current_operations": {},
                                    "created": "20230322T12:05:52Z",
                                    "finished": "20230322T12:25:03Z",
                                    "status": "failure",
                                    "resident_on": "OpaqueRef:259273e3-6fe1-4f9a-b688-ab0847cb81f8",
                                    "progress": 1,
                                    "type": "<none/>",
                                    "result": "",
                                    "error_info": [
                                      "VDI_IO_ERROR",
                                      "Device I/O errors"
                                    ],
                                    "other_config": {},
                                    "subtask_of": "OpaqueRef:NULL",
                                    "subtasks": [],
                                    "backtrace": "(((process xapi)(filename ocaml/xapi/vhd_tool_wrapper.ml)(line 77))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename ocaml/xapi/import_raw_vdi.ml)(line 170)))"
                  
                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates πŸͺ Co-Founder CEO
                    last edited by

                    Device I/O error isn't a good sign. Check dmesg and the disk health.

                    R 2 Replies Last reply Reply Quote 0
                    • R Offline
                      rauly94 @olivierlambert
                      last edited by

                      @olivierlambert sorry to ask, but where should I do this at?

                      AtaxyaNetworkA 1 Reply Last reply Reply Quote 0
                      • AtaxyaNetworkA Offline
                        AtaxyaNetwork Ambassador @rauly94
                        last edited by

                        @rauly94 Hi !

                        You can type "dmesg" directly in your host.
                        You can check your disk's health with the command "smartctl", directly on the host too

                        1 Reply Last reply Reply Quote 0
                        • R Offline
                          rauly94 @olivierlambert
                          last edited by

                          @olivierlambert said in VDI_IO_ERROR(Device I/O errors) when you run scheduled backup:

                          dmesg

                          44519eed-ccdd-4d79-91f1-deea1c1d5013-image.png

                          that's what i got. I'm I doing it correctly.

                          The issue is happening on 1 VM. i tried doing a copy of the same vm and same error. just FYI

                          R 1 Reply Last reply Reply Quote 0
                          • R Offline
                            rauly94 @rauly94
                            last edited by

                            @rauly94 @AtaxyaNetwork

                            R 1 Reply Last reply Reply Quote 0
                            • R Offline
                              rauly94 @rauly94
                              last edited by

                              @rauly94 Hello everyone.
                              Anyone can help me on this issue. Now it started happening on 2 vm's instead of 1 vm. It is happening on the backup replication.

                              Error: VDI_IO_ERROR(Device I/O errors)This is a XenServer/XCP-ng error
                              Start: Jul 5, 2023, 09:03:18 AM
                              End: Jul 5, 2023, 09:44:46 AM
                              Duration: 41 minutes
                              Error: VDI_IO_ERROR(Device I/O errors)This is a XenServer/XCP-ng error

                              Start: Jul 5, 2023, 09:03:09 AM
                              End: Jul 5, 2023, 09:45:12 AM
                              Duration: 42 minutes
                              Error: VDI_IO_ERROR(Device I/O errors)This is a XenServer/XCP-ng error
                              Type: delta

                              M 1 Reply Last reply Reply Quote 1
                              • DanpD Danp forked this topic on
                              • First post
                                Last post