XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Cannot export OVAs

    Scheduled Pinned Locked Moved Solved Xen Orchestra
    64 Posts 7 Posters 7.2k Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • florentF Offline
      florent Vates 🪐 XO Team @florent
      last edited by

      @florent a 10GB export worked
      How many disks have you got in this VM ?

      S A 2 Replies Last reply Reply Quote 0
      • S Offline
        sluflyer06 @florent
        last edited by

        @florent All my VM's only have 1 disk each, my XO vm (10GB) will export but the others range from 40GB up to 100GB and those have never been able to complete.

        1 Reply Last reply Reply Quote 0
        • DanpD Offline
          Danp Pro Support Team
          last edited by

          I just tried exporting a 20GB VM, and it stopped in the middle of the download. Here is what appeared in the logs --

          Apr 24 09:32:42 ubuntuxo xo-server[6111]: 2023-04-24T14:32:42.348Z xo:xo ERROR HTTP request error {
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:   data: { filter: undefined, limit: undefined },
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:   error: NodeError: Premature close
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at ServerResponse.onclose (/opt/xen-orchestra/packages/xo-server/node_modules/readable-stream/lib/internal/streams/end-of-stream.js:116:76)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at ServerResponse.emit (node:events:525:35)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at ServerResponse.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:52:17)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at emitCloseNT (node:_http_server:984:10)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at Socket.onServerResponseClose (node:_http_server:277:5)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at Socket.emit (node:events:525:35)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at Socket.patchedEmit [as emit] (/opt/xen-orchestra/@xen-orchestra/log/configure.js:52:17)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at TCP.<anonymous> (node:net:322:12)
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:       at TCP.callbackTrampoline (node:internal/async_hooks:130:17),
          Apr 24 09:32:42 ubuntuxo xo-server[6111]:   fn: 'handleGetAllObjects'
          Apr 24 09:32:42 ubuntuxo xo-server[6111]: }
          

          The export task is still active / progressing in XO.

          1 Reply Last reply Reply Quote 0
          • DanpD Offline
            Danp Pro Support Team
            last edited by

            Some additional details --

            • OVA export on XOA (latest) worked without any issue.
            • Updated XO from d48ef1f81 to 226f9ad96, and successfully exported the OVA.
            1 Reply Last reply Reply Quote 0
            • A Offline
              Andrew Top contributor @florent
              last edited by Andrew

              @florent Looks like there is a memory leak in XO for the OVA export.

              I'm running current XO source master (226f9ad96). I gave XO 4G (as normal) and it started the OVA export. Then node ran out of memory and started to thrash swap and locked up. I increased the memory to 16G and the same thing happened..... out of memory.

              FYI: I'm trying to export a 120G VM (single disk).

              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                And with XOA there's the same behavior?

                A 1 Reply Last reply Reply Quote 0
                • A Offline
                  Andrew Top contributor @olivierlambert
                  last edited by

                  @florent @olivierlambert Yes. It's a little different, but the end result is the same. XOA VM out of memory... Swap thrash... transfer slows to almost zero.

                  - node: 18.12.1
                  - npm: 8.19.2
                  - xen-orchestra-upload-ova: 0.1.4
                  - xo-server: 5.111.1
                  - xo-server-telemetry: 0.5.0
                  - xo-server-xoa: 0.19.0
                  - xo-web-free: 5.114.0
                  - xoa-cli: 0.35.0
                  - xoa-updater: 0.45.1
                  

                  Also, curl seems to be out of date and does not deploy XOA. Also, the XOA it deploys is old...

                  This worked for me: bash -c "$(wget -qO - https://xoa.io/deploy)" and then I updated to the latest XOA.

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by olivierlambert

                    Not sure to get it, what do you mean by "curl seems to be out of date"?

                    Also, yes, XOA is not re-generated often, eg now we are waiting the next Debian 12 release. So it's normal to update it after deploy 🙂

                    A 1 Reply Last reply Reply Quote 0
                    • A Offline
                      Andrew Top contributor @olivierlambert
                      last edited by

                      @olivierlambert On current XCP 8.2.1:

                      # bash -c "$(curl -sS https://xoa.io/deploy)"
                      curl: (35) Cannot communicate securely with peer: no common encryption algorithm(s).
                      
                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        That's our official doc: https://xen-orchestra.com/docs/xoa.html#via-a-bash-script

                        Maybe it's not updated somewhere else?

                        A 1 Reply Last reply Reply Quote 0
                        • A Offline
                          Andrew Top contributor @olivierlambert
                          last edited by

                          @olivierlambert Curl is from my old notes, so my mistake... It's also still in the older blog posts:

                          https://xen-orchestra.com/blog/one-line-to-deploy-xoa/

                          https://xen-orchestra.com/blog/how-to-start-with-xen-orchestra/

                          1 Reply Last reply Reply Quote 0
                          • olivierlambertO Offline
                            olivierlambert Vates 🪐 Co-Founder CEO
                            last edited by

                            Thanks, fixed in both links 👍

                            1 Reply Last reply Reply Quote 0
                            • florentF Offline
                              florent Vates 🪐 XO Team
                              last edited by

                              @Andrew @sluflyer06 I am exporting VM since yesterday without problems
                              63cbaa0f-e4d7-4d25-86d9-5173142a1f3b-image.png

                              Did you make a yarn or npm install after updating code ? we rollbacked a dependency

                              A 1 Reply Last reply Reply Quote 0
                              • A Offline
                                Andrew Top contributor @florent
                                last edited by

                                @florent I still see the problem.

                                I did a cache clean:

                                yarn cache clean
                                npm cache clean --force
                                

                                and then did a XO source code update and rebuild. OVA export starts downloading at 2MBytes/sec and then fails.

                                I also see the problem on XOA (standard deploy):

                                - node: 18.12.1
                                - npm: 8.19.2
                                - xen-orchestra-upload-ova: 0.1.4
                                - xo-server: 5.111.1
                                - xo-server-telemetry: 0.5.0
                                - xo-server-xoa: 0.19.0
                                - xo-web-free: 5.114.0
                                - xoa-cli: 0.35.0
                                - xoa-updater: 0.45.1
                                

                                Download starts at 10MBytes/second and runs for a while before it fails.

                                florentF 1 Reply Last reply Reply Quote 0
                                • florentF Offline
                                  florent Vates 🪐 XO Team @Andrew
                                  last edited by

                                  @Andrew if this is in a XOA, could you open a support tunnel and allow me to try to download the same ova ?

                                  A 1 Reply Last reply Reply Quote 0
                                  • A Offline
                                    Andrew Top contributor @florent
                                    last edited by

                                    @florent Open.

                                    So, I did not see the failure on a 60G VM, but I do see it on the 120G VM. Both use up memory but it does not hit critical on the smaller VM.

                                    florentF S 2 Replies Last reply Reply Quote 0
                                    • florentF Offline
                                      florent Vates 🪐 XO Team @Andrew
                                      last edited by

                                      @Andrew thanks you for the tunnel I found something

                                      @sluflyer06 can you tests this branch ?
                                      https://github.com/vatesfr/xen-orchestra/pull/6800

                                      fbeauchamp opened this pull request in vatesfr/xen-orchestra

                                      closed fix(xo-vmdk-to-vhd): wait for OVA stream to be written before reading more data #6800

                                      A 1 Reply Last reply Reply Quote 0
                                      • A Offline
                                        Andrew Top contributor @florent
                                        last edited by

                                        @florent That works much better for me! I was able to quickly OVA export VMs that failed before. Memory usage on XO was normal and did not grow. Download speed was good most of the time at 30-35MBytes/sec. It would pause occasionally but toward the end it picked up to 50-60MBytes/sec.

                                        1 Reply Last reply Reply Quote 2
                                        • S Offline
                                          sluflyer06 @Andrew
                                          last edited by

                                          @Andrew Exporting all my large VM's worked on that branch.

                                          A 1 Reply Last reply Reply Quote 1
                                          • A Offline
                                            Andrew Top contributor @sluflyer06
                                            last edited by

                                            @sluflyer06 Today, XO source master has been updated with that fix.

                                            S 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post