XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Exports OVA Timeout...

    Scheduled Pinned Locked Moved Xen Orchestra
    37 Posts 7 Posters 7.7k Views 6 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • florentF Offline
      florent Vates 🪐 XO Team @florent
      last edited by

      @chelle_belle can you test this branch : fix_ova_speed

      there will still have a pause after the 5kb and after each disk, but it should speed up things . I will look for other performance optimization during this sprint.

      A 1 Reply Last reply Reply Quote 0
      • A Offline
        Andrew Top contributor @florent
        last edited by Andrew

        @florent That branch is a LOT faster for OVA export but does not compress the data as much.

        • XVA/NoComp 40 seconds to complete. 1760MB
        • XVA/Zstd 30 seconds to complete. 560MB
        • XVA/gzip 90 seconds to complete. 586MB
        • New OVA 40 seconds to start, 60 seconds to download. 1570MB
        • Old OVA 6 minutes to start, 6 minutes to download. 597MB

        It still leaves a zombie task....

        [XO] VM OVA export (TestSmall7-64 on xcp1) 0%

        1 Reply Last reply Reply Quote 1
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          Thanks for your feedback @Andrew as usual 🙂

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Ah I see, @florent just disabled compression entirely to see the diff, so maybe using compression level to 1 will be better while keeping that speed 🙂

            A 1 Reply Last reply Reply Quote 0
            • A Offline
              Andrew Top contributor @olivierlambert
              last edited by

              @olivierlambert I see that now...

              I guess the best choice would to have an option during export like with XVA. Quick-NoComp OR Slow-Compression. It could also be applied to export disk function.

              I tried 1 but it did not seem to change the size of time. Maybe I did not change it correctly.

              I guess technically the no-compression option could stream the data without the pre-calculation work for compression that takes time but is no longer needed.

              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                Maybe try 6? Don't forget to rebuild after that 🙂

                A 1 Reply Last reply Reply Quote 0
                • A Offline
                  Andrew Top contributor @olivierlambert
                  last edited by Andrew

                  @olivierlambert @florent My mistake.... I did not rebuild XO correctly.

                  Using level 1 is about 1 minute to start and about 1 minute to download and was 637MB. Seems to be a reasonable tradeoff for a significant increase in speed (over level 9) and a minimal decrease in compression.

                  So level 0 (no compression) actually does not help unless the code is modified to just stream the data without the pre-calculation and would cut the export time in half (no scanning/calculation needed).

                  Level 1 is reasonable if it the current procedure is maintained. Level 9 should be optional but not forced. Level 9 is very slow.... If compression is left in place then it would be nice to have some options.... 1..3..6..9... Or some names (min=1, fast=3, standard=6, max=9).

                  The quickest change is to just make it level 1. That still offers compression and still takes time but is MUCH faster and I think that's what most people want. speed...speed....speed....

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    Sounds a good modification to do in the short term until make it editable indeed 🙂

                    @chelle_belle if you need assistance so we can configure it this way on your XOA, let me know 🙂

                    C 1 Reply Last reply Reply Quote 0
                    • C Offline
                      chelle_belle @olivierlambert
                      last edited by

                      @olivierlambert - I'm struggling to switch to the channel and do the update. When I got to set it I get this error message:

                      06/06/2022, 17:07:33: Start updating...
                      06/06/2022, 17:07:33: No manifest found for fix_ova_speed channel.

                      I assumed I had to switch to "Unlisted Channel" and then input fix_ova_speed as the Unlisted channel name...

                      florentF 1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        @chelle_belle a release channel is not the branch on Github, which can be used only if you installed XO from the sources (https://xen-orchestra.com/docs/installation.html#from-the-sources).

                        In theory, we could build packages to match such a release channel, but it's not trivial on our side. It might be easier to just install XO from the sources and not using master branch but fix_ova_speed. If you need assistance on doing that, I'm here 🙂

                        1 Reply Last reply Reply Quote 0
                        • florentF Offline
                          florent Vates 🪐 XO Team @chelle_belle
                          last edited by

                          @chelle_belle If you can open a support tunnel I can deploy the fix today ( with a more reasonable compression level of 1 😅 , thanks @Andrew )

                          Can you open a support ticket with the tunnel information ?

                          1 Reply Last reply Reply Quote 0
                          • A Offline
                            Andrew Top contributor
                            last edited by

                            @florent @julien-f Trying the new OVA export (XO source commit c11e0)..... I exported a VM quickly and without a zombie task left over (good), but then I tried to import it and got no error when clicking import but it did nothing (failure without notice). The logs do show an error:

                            HTTP handler of vm.import
                            undefined
                            {
                              "code": "INTERNAL_ERROR",
                              "params": [
                                "(Failure \"Expected string, got 'N'\")"
                              ],
                              "call": {
                                "method": "VM.create",
                                "params": [
                                  {
                                    "actions_after_crash": "restart",
                                    "actions_after_reboot": "restart",
                                    "actions_after_shutdown": "destroy",
                                    "affinity": null,
                                    "HVM_boot_params": {
                                      "order": "cdn"
                                    },
                                    "HVM_boot_policy": "BIOS order",
                                    "is_a_template": false,
                                    "memory_dynamic_max": 4294967296,
                                    "memory_dynamic_min": 4294967296,
                                    "memory_static_max": 4294967296,
                                    "memory_static_min": 4294967296,
                                    "other_config": {
                                      "vgpu_pci": "",
                                      "base_template_name": "Other install media",
                                      "install-methods": "cdrom"
                                    },
                                    "PCI_bus": "",
                                    "platform": {
                                      "timeoffset": "0",
                                      "nx": "true",
                                      "acpi": "1",
                                      "apic": "true",
                                      "pae": "true",
                                      "hpet": "true",
                                      "viridian": "true"
                                    },
                                    "PV_args": "",
                                    "PV_bootloader_args": "",
                                    "PV_bootloader": "",
                                    "PV_kernel": "",
                                    "PV_legacy_args": "",
                                    "PV_ramdisk": "",
                                    "recommendations": "<restrictions><restriction field=\"memory-static-max\" max=\"137438953472\" /><restriction field=\"vcpus-max\" max=\"32\" /><restriction property=\"number-of-vbds\" max=\"255\" /><restriction property=\"number-of-vifs\" max=\"7\" /><restriction field=\"has-vendor-device\" value=\"false\" /></restrictions>",
                                    "user_version": 1,
                                    "VCPUs_at_startup": 2,
                                    "VCPUs_max": 2,
                                    "VCPUs_params": {},
                                    "blocked_operations": {},
                                    "has_vendor_device": false,
                                    "HVM_shadow_multiplier": 1,
                                    "name_description": "",
                                    "name_label": "NginX Test (ova)",
                                    "order": 0,
                                    "shutdown_delay": 0,
                                    "start_delay": 0,
                                    "version": 0
                                  }
                                ]
                              },
                              "message": "INTERNAL_ERROR((Failure \"Expected string, got 'N'\"))",
                              "name": "XapiError",
                              "stack": "XapiError: INTERNAL_ERROR((Failure \"Expected string, got 'N'\"))
                                at Function.wrap (/opt/xo/xo-builds/xen-orchestra-202206091200/packages/xen-api/src/_XapiError.js:16:12)
                                at /opt/xo/xo-builds/xen-orchestra-202206091200/packages/xen-api/src/transports/json-rpc.js:37:27
                                at AsyncResource.runInAsyncScope (node:async_hooks:202:9)
                                at cb (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/util.js:355:42)
                                at tryCatcher (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/util.js:16:23)
                                at Promise._settlePromiseFromHandler (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/promise.js:547:31)
                                at Promise._settlePromise (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/promise.js:604:18)
                                at Promise._settlePromise0 (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/promise.js:649:10)
                                at Promise._settlePromises (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/promise.js:729:18)
                                at _drainQueueStep (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/async.js:93:12)
                                at _drainQueue (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/async.js:86:9)
                                at Async._drainQueues (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/async.js:102:5)
                                at Immediate.Async.drainQueues [as _onImmediate] (/opt/xo/xo-builds/xen-orchestra-202206091200/node_modules/bluebird/js/release/async.js:15:14)
                                at processImmediate (node:internal/timers:466:21)
                                at process.callbackTrampoline (node:internal/async_hooks:130:17)"
                            }
                            
                            julien-fJ 2 Replies Last reply Reply Quote 0
                            • julien-fJ Offline
                              julien-f Vates 🪐 Co-Founder XO Team @Andrew
                              last edited by

                              @Andrew This problem does not happen before the last 3 commits?

                              I believe the issue is the affinity: null, but I don't see how it can be related to these changes. 🤔

                              julien-fJ 1 Reply Last reply Reply Quote 0
                              • julien-fJ Offline
                                julien-f Vates 🪐 Co-Founder XO Team @julien-f
                                last edited by

                                Probably related to https://github.com/vatesfr/xen-orchestra/commit/2af5328a0f69912f5c03f5a8894fd75633bd3655

                                0 julien-f committed to vatesfr/xen-orchestra
                                chore(xo-server): use @xen-orchestra/xapi/VM_create
                                1 Reply Last reply Reply Quote 0
                                • julien-fJ Offline
                                  julien-f Vates 🪐 Co-Founder XO Team @Andrew
                                  last edited by

                                  @Andrew Should be fixed, please test and keep me posted 🙂

                                  https://github.com/vatesfr/xen-orchestra/commit/83690a4dd4457daf9ea402e623c9f3fe9ce9ab7f

                                  0 julien-f committed to vatesfr/xen-orchestra
                                  fix(xo-server/_importOvaVm): fix VM creation
                                  
                                  Fixes https://xcp-ng.org/forum/post/49920
                                  
                                  Introduced by 2af5328a0f69912f5c03f5a8894fd75633bd3655
                                  A 1 Reply Last reply Reply Quote 0
                                  • A Offline
                                    Andrew Top contributor @julien-f
                                    last edited by

                                    @julien-f Still an issue... Two steps forward, one step back (commit 83690).

                                    Import now starts, no error, but it does not finish.

                                    The task is started, the VM is created, the data is uploaded, the task exits, but the import page does not finish and the VM name is left as [Importing...] NginX Test. No logs in the UI.

                                    I can force start the VM, so it looks like the data is ok.

                                    florentF 1 Reply Last reply Reply Quote 0
                                    • florentF Offline
                                      florent Vates 🪐 XO Team @Andrew
                                      last edited by

                                      @Andrew I am reproducing the problem : if the VM has mulitple disks, the import only handle the first one and freeze

                                      A 1 Reply Last reply Reply Quote 1
                                      • olivierlambertO Offline
                                        olivierlambert Vates 🪐 Co-Founder CEO
                                        last edited by

                                        @Andrew do you have more than 1 disk in your test VM?

                                        A 1 Reply Last reply Reply Quote 0
                                        • A Offline
                                          Andrew Top contributor @olivierlambert
                                          last edited by

                                          @olivierlambert No. Just one disk.

                                          1 Reply Last reply Reply Quote 0
                                          • A Offline
                                            Andrew Top contributor @florent
                                            last edited by

                                            @florent The only thing I see in the log is: yielded last VMDK block

                                            A 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post