XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Our future backup code: test it!

    Scheduled Pinned Locked Moved Backup
    77 Posts 10 Posters 2.6k Views 10 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Tristis OrisT Online
      Tristis Oris Top contributor
      last edited by

      same error as above

      @xen-orchestra/disk-transform:build: cache miss, executing 6b8daba7020cf148
      @xen-orchestra/disk-transform:build: yarn run v1.22.22
      @xen-orchestra/disk-transform:build: $ tsc
      @xen-orchestra/disk-transform:build: src/SynchronizedDisk.mts(2,30): error TS2307: Cannot find module '@vates/generator-toolbox' or its corresponding type declarations.
      @xen-orchestra/disk-transform:build: error Command failed with exit code 2.
      @xen-orchestra/disk-transform:build: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
      @xen-orchestra/disk-transform:build: ERROR: command finished with error: command (/opt/xo/xo-builds/xen-orchestra-202504071027/@xen-orchestra/disk-transform) /tmp/yarn--1744010914016-0.7668824784043369/yarn run build exited (2)
      @xen-orchestra/disk-transform#build: command (/opt/xo/xo-builds/xen-orchestra-202504071027/@xen-orchestra/disk-transform) /tmp/yarn--1744010914016-0.7668824784043369/yarn run build exited (2)
      
       Tasks:    24 successful, 28 total
      Cached:    0 cached, 28 total
        Time:    11.251s
      Failed:    @xen-orchestra/disk-transform#build
      
       ERROR  run failed: command  exited (2)
      error Command failed with exit code 2.
      info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
      + rm -rf /opt/xo/xo-builds/xen-orchestra-202504071027
      
      1 Reply Last reply Reply Quote 0
      • florentF Offline
        florent Vates 🪐 XO Team
        last edited by

        @Tristis-Oris did you run yarn from the root of the project to install the new dependencies ?

        Tristis OrisT 1 Reply Last reply Reply Quote 0
        • Tristis OrisT Online
          Tristis Oris Top contributor @florent
          last edited by Tristis Oris

          @florent

          yarn install v1.22.22
          [1/5] Validating package.json...
          [2/5] Resolving packages...
          [3/5] Fetching packages...
          warning Pattern ["@intlify/shared@latest"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-@intlify-shared-11.1.2-d780bc8eb4e3c3ef8a3e45c421f0f5ecf59651f2-integrity/node_modules/@intlify/shared" as pattern ["@intlify/shared@11.1.2","@intlify/shared@11.1.2","@intlify/shared@11.1.2"]. This could result in non-deterministic behavior, skipping.
          warning Pattern ["@intlify/shared@next"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-@intlify-shared-11.0.0-rc.1-52a67aa12fccd9303027b48ebf017b75b7350283-integrity/node_modules/@intlify/shared" as pattern ["@intlify/shared@11.0.0-rc.1"]. This could result in non-deterministic behavior, skipping.
          warning bare-os@3.4.0: The engine "bare" appears to be invalid.
          [4/5] Linking dependencies...
          warning "@commitlint/cli > @commitlint/load > cosmiconfig-typescript-loader@6.1.0" has unmet peer dependency "@types/node@*".
          warning "@commitlint/cli > @commitlint/load > cosmiconfig-typescript-loader@6.1.0" has unmet peer dependency "typescript@>=5".
          warning "@typescript-eslint/eslint-plugin > ts-api-utils@1.4.3" has unmet peer dependency "typescript@>=4.2.0".
          warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "pinia@^3.0.1".
          warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "vue@~3.5.13".
          warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "vue-i18n@^11.1.2".
          warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "vue-router@^4.5.0".
          warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @xen-orchestra/rest-api > inversify@6.2.2" has unmet peer dependency "reflect-metadata@~0.2.2".
          warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @xen-orchestra/rest-api > swagger-ui-express@5.0.1" has unmet peer dependency "express@>=4.0.0 || >=5.0.0-beta".
          warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @vates/event-listeners-manager > tap > @tapjs/test > @isaacs/ts-node-temp-fork-for-pr-2009@10.9.7" has unmet peer dependency "@types/node@*".
          warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @xen-orchestra/rest-api > inversify > @inversifyjs/core > @inversifyjs/reflect-metadata-utils@0.2.4" has unmet peer dependency "reflect-metadata@0.2.2".
          warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @vates/event-listeners-manager > tap > @tapjs/asserts > tcompare > react-element-to-jsx-string@15.0.0" has unmet peer dependency "react@^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1 || ^18.0.0".
          warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @vates/event-listeners-manager > tap > @tapjs/asserts > tcompare > react-element-to-jsx-string@15.0.0" has unmet peer dependency "react-dom@^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1 || ^18.0.0".
          warning Workspaces can only be enabled in private projects.
          [5/5] Building fresh packages...
          $ husky install
          husky - Git hooks installed
          Done in 39.86s.
          
          

          but same error.

          florentF 1 Reply Last reply Reply Quote 0
          • florentF Offline
            florent Vates 🪐 XO Team @Tristis Oris
            last edited by

            @Tristis-Oris it looks like the build from completly empty repo don't work as expected

            can you try this :

            cd @vates/generator-toolbox/
            yarn build 
            cd - 
            yarn build
            

            I am working on improving the build process in the meantimes

            Tristis OrisT 1 Reply Last reply Reply Quote 0
            • Tristis OrisT Online
              Tristis Oris Top contributor @florent
              last edited by

              @florent yep it works now.

              Tristis OrisT 1 Reply Last reply Reply Quote 0
              • J Offline
                john.c
                last edited by

                So is it this new backup code which the tape back up support will be plugging into?

                If so I have been thinking may be have it when doing the tape backup support, get the XOA to install the packages when required after prompting the user. Also have xo-server instruct the creation of a new disk image for attaching to xoa. The new disk image would act as storage or cache for the extra data used by tape drive backups.

                The tape drive related software wouldn't be installed by default, unless specified as such during deployment with the XOA deploy tool. But instead installed at user request, by selecting to use tape backup destination when setting up back up job, clicking on the shim widget about tape backups or restoring from an online Vates account backup of XOA settings.

                florentF 1 Reply Last reply Reply Quote 0
                • D Offline
                  Davidj 0 @florent
                  last edited by Davidj 0

                  @florent I'm doing this from source.
                  Running git pull --rebase seems to have fixed the build problems.

                  1 Reply Last reply Reply Quote 2
                  • florentF Offline
                    florent Vates 🪐 XO Team @john.c
                    last edited by florent

                    @john.c the backup on tape is still on the roadmap but won't be for this step

                    The main change is that we'll have to rework part of the code to ensure we write data sequentially + the management tools ( which backup is on which tape )

                    You can try the FS on tape , but the performance penalty is heavy

                    J 1 Reply Last reply Reply Quote 0
                    • J Offline
                      john.c @florent
                      last edited by

                      @florent said in Our future backup code: test it!:

                      @john.c the backup on tape is still on the roadmap but won't be for this step

                      The main change is that we'll have to rework part of the code to ensure we write data sequentially + the management tools ( which backup is on which tape )

                      You can try the FS on tape , but the performance penalty is heavy

                      Even if its not part of this rewrite some of what I wrote is my idea or view as to how it could be implemented later on. So that it's implemented in a way, that permits people who don't need tape back ups to still have small Xen Orchestra instances. But at the same time enable those who use tape backups, to have the capacity and an appropriately sized Xen Orchestra.

                      1 Reply Last reply Reply Quote 0
                      • D Offline
                        Davidj 0
                        last edited by

                        Testing a continuous replication backup, I get Error: can't connect to any nbd client

                        Running the same job on my regular XO works fine. Let me know what logs you want to see.

                        mail (h2)
                        
                           Snapshot
                           Start: 2025-04-10 18:18
                           End: 2025-04-10 18:18
                           h3 local storage (130.02 GiB free) - h3
                           Start: 2025-04-10 18:18
                        
                        Start: 2025-04-10 18:18
                        End: 2025-04-10 18:18
                        Duration: a few seconds
                        Error: can't connect to any nbd client
                        Type: delta
                        
                        1 Reply Last reply Reply Quote 0
                        • Tristis OrisT Online
                          Tristis Oris Top contributor @Tristis Oris
                          last edited by

                          @Tristis-Oris probably after https://github.com/vatesfr/xen-orchestra/pull/8432/files/04be6032180e7245a8db9c5f78177ac7999882c2

                          /@vates/generator-toolbox# yarn build
                          yarn run v1.22.22
                          $ tsc
                          /bin/sh: 1: tsc: not found
                          error Command failed with exit code 127.
                          info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
                          

                          ok, npm install -g typescript

                          yarn build
                          yarn run v1.22.22
                          $ tsc
                          src/synchronized.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.
                          
                          1 import assert from 'node:assert'
                                               ~~~~~~~~~~~~~
                          
                          src/synchronized.test.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.
                          
                          1 import assert from 'node:assert'
                                               ~~~~~~~~~~~~~
                          
                          src/synchronized.test.mts:2:29 - error TS2307: Cannot find module 'node:test' or its corresponding type declarations.
                          
                          2 import { suite, test } from 'node:test'
                                                        ~~~~~~~~~~~
                          
                          src/throttle.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.
                          
                          1 import assert from 'node:assert'
                                               ~~~~~~~~~~~~~
                          
                          src/throttle.test.mts:2:29 - error TS2307: Cannot find module 'node:test' or its corresponding type declarations.
                          
                          2 import { suite, test } from 'node:test'
                                                        ~~~~~~~~~~~
                          
                          src/throttle.test.mts:3:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.
                          
                          3 import assert from 'node:assert'
                                               ~~~~~~~~~~~~~
                          
                          src/throttle.test.mts:7:11 - error TS2580: Cannot find name 'Buffer'. Do you need to install type definitions for node? Try `npm i --save-dev @types/node`.
                          
                          7     yield Buffer.allocUnsafe(chunkSize)
                                      ~~~~~~
                          
                          src/timeout.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.
                          
                          1 import assert from 'node:assert'
                                               ~~~~~~~~~~~~~
                          
                          src/timeout.test.mts:1:30 - error TS2307: Cannot find module 'node:test' or its corresponding type declarations.
                          
                          1 import { describe, it } from 'node:test'
                                                         ~~~~~~~~~~~
                          
                          src/timeout.test.mts:2:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.
                          
                          2 import assert from 'node:assert'
                                               ~~~~~~~~~~~~~
                          
                          
                          Found 10 errors in 6 files.
                          
                          Errors  Files
                               1  src/synchronized.mts:1
                               2  src/synchronized.test.mts:1
                               1  src/throttle.mts:1
                               3  src/throttle.test.mts:2
                               1  src/timeout.mts:1
                               2  src/timeout.test.mts:1
                          error Command failed with exit code 2.
                          info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
                          
                          fbeauchamp opened this pull request in vatesfr/xen-orchestra

                          closed feat(backups): use generator instead of streams for backup and replication #8432

                          Tristis OrisT 1 Reply Last reply Reply Quote 0
                          • Tristis OrisT Online
                            Tristis Oris Top contributor @Tristis Oris
                            last edited by

                            @Tristis-Oris
                            from root directory.
                            /xen-orchestra
                            yarn - succeess
                            yarn build - failed
                            again to cd @vates/generator-toolbox/
                            yarn build

                            yarn run v1.22.22
                            $ tsc
                            Done in 5.50s.
                            

                            cd -
                            yarn build

                             Tasks:    29 successful, 29 total
                            Cached:    13 cached, 29 total
                              Time:    1m17.02s
                            
                            Done in 77.34s.
                            

                            again at root

                            cd packages/xo-server
                            mkdir -p ~/.config/xo-server
                            cp sample.config.toml ~/.config/xo-server/config.toml
                            yarn start
                            
                            1 Reply Last reply Reply Quote 0
                            • Tristis OrisT Online
                              Tristis Oris Top contributor
                              last edited by

                              delta, first full.

                              • no NBD
                              2025-04-11T12:19:15.169Z xo:backups:worker INFO starting backup
                              2025-04-11T12:22:29.509Z xo:backups:MixinBackupWriter WARN cleanVm: incorrect backup size in metadata {
                                path: '/xo-vm-backups/07e5d245-7ed3-b690-e386-3461556ad41f/20250411T121931Z.json',
                                actual: 27271364608,
                                expected: 27278188544
                              }
                              

                              2 vms but 1 log at cli.

                              delta

                              [2025-04-11T12:27:41.428Z xo:backups:worker INFO starting backup
                              2025-04-11T12:27:48.644Z xo:backups:MixinBackupWriter WARN cleanVm: incorrect backup size in metadata {
                                path: '/xo-vm-backups/f3cfdc19-0e6a-04d1-5651-82e1dbb25191/20250411T122746Z.json',
                                actual: 44040192,
                                expected: 46314496
                              }
                              2025-04-11T12:28:03.055Z xo:backups:MixinBackupWriter WARN cleanVm: incorrect backup size in metadata {
                                path: '/xo-vm-backups/07e5d245-7ed3-b690-e386-3461556ad41f/20250411T122801Z.json',
                                actual: 46137344,
                                expected: 48412160
                              }
                              2025-04-11T12:28:03.081Z xo:backups:worker INFO backup has ended
                              2025-04-11T12:28:03.181Z xo:backups:worker INFO process will exit {
                                duration: 21752192,
                                exitCode: 0,
                                resourceUsage: {
                                  userCPUTime: 3794057,
                                  systemCPUTime: 468414,
                                  maxRSS: 69592,
                                  sharedMemorySize: 0,
                                  unsharedDataSize: 0,
                                  unsharedStackSize: 0,
                                  minorPageFault: 42277,
                                  majorPageFault: 0,
                                  swappedOut: 0,
                                  fsRead: 13624,
                                  fsWrite: 185840,
                                  ipcSent: 0,
                                  ipcReceived: 0,
                                  signalsCount: 0,
                                  voluntaryContextSwitches: 3591,
                                  involuntaryContextSwitches: 441
                                },
                                summary: { duration: '22s', cpuUsage: '20%', memoryUsage: '67.96 MiB' }
                              }
                              

                              can't get NBD to work. It enabled on nic, but not used for backups. XO settings copied from prod instance.

                              1 Reply Last reply Reply Quote 0
                              • D Offline
                                Davidj 0
                                last edited by

                                Like @Tristis-Oris , my first tests were done using a clone of my production XO. They failed with "No NBD" errors.

                                Creating a new backup works.

                                Tristis OrisT 1 Reply Last reply Reply Quote 0
                                • Tristis OrisT Online
                                  Tristis Oris Top contributor @Davidj 0
                                  last edited by

                                  @Davidj-0 No NBD usage even with new task( Main XO is on same host, same nic.

                                  D 1 Reply Last reply Reply Quote 0
                                  • D Offline
                                    Davidj 0 @Tristis Oris
                                    last edited by

                                    @Tristis-Oris My setup is similar to yours, and I get similar errors on any existing backup job.

                                    However, if I create a new backup job, then it works without any error.

                                    ping @florent , maybe these data points are useful.

                                    1 Reply Last reply Reply Quote 0
                                    • florentF Offline
                                      florent Vates 🪐 XO Team
                                      last edited by

                                      thanks for your tests . I will recheck NBD backup

                                      florentF 1 Reply Last reply Reply Quote 0
                                      • florentF Offline
                                        florent Vates 🪐 XO Team @florent
                                        last edited by

                                        ih @Tristis-Oris and @Davidj-0 I pushed an update to better handle NBD error, please keep us informed

                                        Tristis OrisT 1 Reply Last reply Reply Quote 0
                                        • Tristis OrisT Online
                                          Tristis Oris Top contributor @florent
                                          last edited by Tristis Oris

                                          @florent i'm not sure how to update from this branch(

                                          git checkout .
                                          git pull --ff-only
                                          yarn
                                          yarn build
                                          
                                          Updating 04be60321..6641580e6
                                          Fast-forward
                                           @xen-orchestra/xapi/disks/Xapi.mjs             | 17 ++++++++++-------
                                           @xen-orchestra/xapi/disks/XapiVhdCbt.mjs       |  3 ++-
                                           @xen-orchestra/xapi/disks/XapiVhdStreamNbd.mjs |  3 ---
                                           @xen-orchestra/xapi/disks/utils.mjs            | 29 ++++++++++++++++-------------
                                           4 files changed, 28 insertions(+), 24 deletions(-)
                                          
                                          git checkout feat_generator_backups
                                          Already on 'feat_generator_backups'
                                          Your branch is up to date with 'origin/feat_generator_backups'.
                                          
                                          
                                          git pull origin feat_generator_backups
                                          From https://github.com/vatesfr/xen-orchestra
                                           * branch                feat_generator_backups -> FETCH_HEAD
                                          Already up to date.
                                          

                                          but still no NBD at backup.

                                          1 Reply Last reply Reply Quote 0
                                          • Tristis OrisT Online
                                            Tristis Oris Top contributor
                                            last edited by

                                            i see significant speed difference from master branch.

                                            prod
                                            NBD
                                            160.71 MiB
                                            no NBD
                                            169

                                            test
                                            NBD (broken)
                                            110-130 MiB
                                            no NBD
                                            115

                                            florentF 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post