XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Continuous Replication fails with VDI_IO_ERROR(Device I/O errors) on fresh XOA install after XenServer 7.2 -> XCP-ng 8.0 upgrade

    Scheduled Pinned Locked Moved Solved Xen Orchestra
    20 Posts 7 Posters 4.8k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 1 Offline
      1am3r
      last edited by olivierlambert

      Hi,
      we upgraded our XenServer 7.2 pool to XCP-ng 8.0.

      We previously hat Continuous Replication from a iSCSI storage to a NFS Storage running.

      After the update, all replication jobs fail with VDI_IO_ERROR(Device I/O errors) and no apparent erros in SMlog or xensource.log (or I don't know for what to look for).

      Is this a known problem? How can we further debug this problem?

      Thanks in advance,
      Michael

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Online
        olivierlambert Vates πŸͺ Co-Founder CEO
        last edited by

        Hi,

        I'm not aware of that issue. You could try to add the following entry in your Xen Orchestra configuration and try again:

        guessVhdSizeOnImport: true
        

        If you have XOA, we can do that remotely for you.

        1 Reply Last reply Reply Quote 0
        • 1 Offline
          1am3r
          last edited by

          Changing this setting fixed the replication in the first tests. Are there any negative side effects with this?

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Online
            olivierlambert Vates πŸͺ Co-Founder CEO
            last edited by

            Keep us posted after few runs (to be sure it's not due to a first full that it worked). You shouldn't have any downside using it, it's a setting we didn't put by default until we are 100% sure of the outcome (especially on retrocompatibility for older hosts)

            1 Reply Last reply Reply Quote 0
            • 1 Offline
              1am3r
              last edited by

              It looks good for now. I'll monitor it for a while and report back. Thanks!

              1 Reply Last reply Reply Quote 0
              • olivierlambertO Online
                olivierlambert Vates πŸͺ Co-Founder CEO
                last edited by

                You are welcome πŸ™‚

                1 Reply Last reply Reply Quote 0
                • N Offline
                  nyash
                  last edited by nyash

                  Hi,

                  I had noticed a similar problem. I have upgraded from XCP-NG 7.4 to 8.0 via the ISO. Upgrade was painless. Only issue was with a single VM not starting that was resolved with "xe vm-cd-eject --multiple".

                  I have also upgraded to the latest Xen Orchestra from sources:
                  :xo-server 5.46.0
                  :xo-web 5.46.0

                  I have 2 xenservers with local storage defined like this:
                  https://i.imgur.com/XCCyzS2.png

                  I have defined a simple Continous Replication task like this to backup a very small VM from xenserver1 to xenserver2:
                  https://i.imgur.com/d8nuDAN.png

                  And started the task manually, however it eventually fails like this:
                  https://i.imgur.com/B5IS5lB.png

                  Attached log:
                  https://pastebin.com/raw/gS9SYXgg

                  Changing the setting in config.toml of xo-server to: 'guessVhdSizeOnImport: true' seems to have fixed the problem (the backup task completes succesfully).

                  Thanks.

                  1 Reply Last reply Reply Quote 0
                  • rizaemet 0R Offline
                    rizaemet 0
                    last edited by

                    Same problem here after upgrade xcp-ng 7.6 to 8.0

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Online
                      olivierlambert Vates πŸͺ Co-Founder CEO
                      last edited by

                      And same solution worked?

                      rizaemet 0R 1 Reply Last reply Reply Quote 1
                      • rizaemet 0R Offline
                        rizaemet 0 @olivierlambert
                        last edited by

                        @olivierlambert I've just tried and yes, 'guessVhdSizeOnImport: true' solved the problem.

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO Online
                          olivierlambert Vates πŸͺ Co-Founder CEO
                          last edited by

                          Okay I'm pinging @julien-f so we can include this as default (because it seems to not break for older XS)

                          julien-fJ 1 Reply Last reply Reply Quote 0
                          • 1 Offline
                            1am3r
                            last edited by

                            We run this config now for a week and didn't have any problems. So it looks good from our site.

                            1 Reply Last reply Reply Quote 2
                            • txsastreT Offline
                              txsastre
                              last edited by

                              same problem, same solution. works fine πŸ˜‰

                              1 Reply Last reply Reply Quote 0
                              • rizaemet 0R Offline
                                rizaemet 0
                                last edited by rizaemet 0

                                'guessVhdSizeOnImport: true' looks like solved the problem 9 day ago. But I'm getting same error sometimes and randomly since that day.
                                Daily CR job have 31 VMs. Some day successful 31/31. But some days one or two VM replication fail with tis message: VDI_IO_ERROR(Device I/O errors).
                                I think my problem still related with this topic.

                                N 1 Reply Last reply Reply Quote 0
                                • N Offline
                                  nyash @rizaemet 0
                                  last edited by nyash

                                  @rizaemet-0
                                  Ever since I have changed to guessVhdSizeOnImport: true I have not had any problems. However I only backup 5 VMs which is not much.

                                  In the past I noticed that if I didn't change the default "Concurrency" setting to 1 in the Backup NGs advanced settings (I assume that it tries to parallelize backups by default) then my tasks would fail randomly once in a while with VDI_IO_ERROR(Device I/O errors). I figured that perhaps my servers are not fast enough; they have about 40MB/s write speed to the disks only. Setting to 1 solved it, but I can imagine this may not be desirable if there is lots of VMs to backup since it would slow down the process considerably.

                                  1 Reply Last reply Reply Quote 0
                                  • julien-fJ Offline
                                    julien-f Vates πŸͺ Co-Founder XO Team @olivierlambert
                                    last edited by

                                    @olivierlambert Yep, I think it's safe enough to turn on by default in the next release πŸ™‚

                                    1 Reply Last reply Reply Quote 1
                                    • olivierlambertO Online
                                      olivierlambert Vates πŸͺ Co-Founder CEO
                                      last edited by

                                      Great, so let's do that!

                                      1 Reply Last reply Reply Quote 1
                                      • onurO Offline
                                        onur
                                        last edited by

                                        @olivierlambert said in Continuous Replication fails with VDI_IO_ERROR(Device I/O errors) on fresh XOA install after XenServer 7.2 -> XCP-ng 8.0 upgrade:

                                        Xen Orchestra configuration

                                        should I add "guessVhdSizeOnImport: true" to /opt/xen-orchestra/packages/xo-server/config.toml

                                        1 Reply Last reply Reply Quote 0
                                        • olivierlambertO Online
                                          olivierlambert Vates πŸͺ Co-Founder CEO
                                          last edited by olivierlambert

                                          To your config file, yes. Where this config file is depends on where do you installed Xen Orchestra (I suppose from the sources, so I can't guess where do you installed, could be anyware πŸ˜‰ )

                                          edit: I think it's fixed now and integrated by default in the config

                                          onurO 1 Reply Last reply Reply Quote 1
                                          • onurO Offline
                                            onur @olivierlambert
                                            last edited by

                                            @olivierlambert said in Continuous Replication fails with VDI_IO_ERROR(Device I/O errors) on fresh XOA install after XenServer 7.2 -> XCP-ng 8.0 upgrade:

                                            To your config file, yes. Where this config file is depends on where do you installed Xen Orchestra (I suppose from the sources, so I can't guess where do you installed, could be anyware πŸ˜‰ )

                                            edit: I think it's fixed now and integrated by default in the config

                                            Great, this fix already is in the new version.
                                            I was having this problem (Error: VDI_IO_ERROR(Device I/O errors)) and was not be able to do CR of my VMs but now CR works.
                                            Thank you

                                            PS: BTW that was my config file πŸ˜‰

                                            1 Reply Last reply Reply Quote 0

                                            Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                            Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                            With your input, this post could be even better πŸ’—

                                            Register Login
                                            • First post
                                              Last post