Posts made by Tristis Oris | XCP-ng and XO forum

Tristis Oris

@florent
same speed after fix.
142.56 MiB/s - 145.63 MiB/s

maybe i miss something else? same database as prod, only another LUN connected for backups.

Tristis Oris

@florent blocks! i forgot about that checkbox)

better but not same.

Duration: 3 minutes
Size: 26.53 GiB
Speed: 143.7 MiB/s

Speed: 146.09 MiB/s

Tristis Oris

@florent

1 vm, 1 storage, NBD connections: 1. delta, first full.

Duration: 3 minutes
Size: 26.54 GiB
Speed: 160.71 MiB/s

Duration: 4 minutes
Size: 26.53 GiB
Speed: 113.74 MiB/s

Tristis Oris

i see significant speed difference from master branch.

prod
NBD
160.71 MiB
no NBD
169

test
NBD (broken)
110-130 MiB
no NBD
115

Tristis Oris

@florent i'm not sure how to update from this branch(

git checkout .
git pull --ff-only
yarn
yarn build

Updating 04be60321..6641580e6
Fast-forward
 @xen-orchestra/xapi/disks/Xapi.mjs             | 17 ++++++++++-------
 @xen-orchestra/xapi/disks/XapiVhdCbt.mjs       |  3 ++-
 @xen-orchestra/xapi/disks/XapiVhdStreamNbd.mjs |  3 ---
 @xen-orchestra/xapi/disks/utils.mjs            | 29 ++++++++++++++++-------------
 4 files changed, 28 insertions(+), 24 deletions(-)

git checkout feat_generator_backups
Already on 'feat_generator_backups'
Your branch is up to date with 'origin/feat_generator_backups'.


git pull origin feat_generator_backups
From https://github.com/vatesfr/xen-orchestra
 * branch                feat_generator_backups -> FETCH_HEAD
Already up to date.

but still no NBD at backup.

Tristis Oris

Merged. Works as intented, thank you.

Tristis Oris

@Davidj-0 No NBD usage even with new task( Main XO is on same host, same nic.

Tristis Oris

delta, first full.

no NBD

2025-04-11T12:19:15.169Z xo:backups:worker INFO starting backup
2025-04-11T12:22:29.509Z xo:backups:MixinBackupWriter WARN cleanVm: incorrect backup size in metadata {
  path: '/xo-vm-backups/07e5d245-7ed3-b690-e386-3461556ad41f/20250411T121931Z.json',
  actual: 27271364608,
  expected: 27278188544
}

2 vms but 1 log at cli.

delta

[2025-04-11T12:27:41.428Z xo:backups:worker INFO starting backup
2025-04-11T12:27:48.644Z xo:backups:MixinBackupWriter WARN cleanVm: incorrect backup size in metadata {
  path: '/xo-vm-backups/f3cfdc19-0e6a-04d1-5651-82e1dbb25191/20250411T122746Z.json',
  actual: 44040192,
  expected: 46314496
}
2025-04-11T12:28:03.055Z xo:backups:MixinBackupWriter WARN cleanVm: incorrect backup size in metadata {
  path: '/xo-vm-backups/07e5d245-7ed3-b690-e386-3461556ad41f/20250411T122801Z.json',
  actual: 46137344,
  expected: 48412160
}
2025-04-11T12:28:03.081Z xo:backups:worker INFO backup has ended
2025-04-11T12:28:03.181Z xo:backups:worker INFO process will exit {
  duration: 21752192,
  exitCode: 0,
  resourceUsage: {
    userCPUTime: 3794057,
    systemCPUTime: 468414,
    maxRSS: 69592,
    sharedMemorySize: 0,
    unsharedDataSize: 0,
    unsharedStackSize: 0,
    minorPageFault: 42277,
    majorPageFault: 0,
    swappedOut: 0,
    fsRead: 13624,
    fsWrite: 185840,
    ipcSent: 0,
    ipcReceived: 0,
    signalsCount: 0,
    voluntaryContextSwitches: 3591,
    involuntaryContextSwitches: 441
  },
  summary: { duration: '22s', cpuUsage: '20%', memoryUsage: '67.96 MiB' }
}

can't get NBD to work. It enabled on nic, but not used for backups. XO settings copied from prod instance.

Tristis Oris

@Tristis-Oris
from root directory.
/xen-orchestra
yarn - succeess
yarn build - failed
again to cd @vates/generator-toolbox/
yarn build

yarn run v1.22.22
$ tsc
Done in 5.50s.

cd -
yarn build

 Tasks:    29 successful, 29 total
Cached:    13 cached, 29 total
  Time:    1m17.02s

Done in 77.34s.

again at root

cd packages/xo-server
mkdir -p ~/.config/xo-server
cp sample.config.toml ~/.config/xo-server/config.toml
yarn start

Tristis Oris

@Tristis-Oris probably after https://github.com/vatesfr/xen-orchestra/pull/8432/files/04be6032180e7245a8db9c5f78177ac7999882c2

/@vates/generator-toolbox# yarn build
yarn run v1.22.22
$ tsc
/bin/sh: 1: tsc: not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

ok, npm install -g typescript

yarn build
yarn run v1.22.22
$ tsc
src/synchronized.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.

1 import assert from 'node:assert'
                     ~~~~~~~~~~~~~

src/synchronized.test.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.

1 import assert from 'node:assert'
                     ~~~~~~~~~~~~~

src/synchronized.test.mts:2:29 - error TS2307: Cannot find module 'node:test' or its corresponding type declarations.

2 import { suite, test } from 'node:test'
                              ~~~~~~~~~~~

src/throttle.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.

1 import assert from 'node:assert'
                     ~~~~~~~~~~~~~

src/throttle.test.mts:2:29 - error TS2307: Cannot find module 'node:test' or its corresponding type declarations.

2 import { suite, test } from 'node:test'
                              ~~~~~~~~~~~

src/throttle.test.mts:3:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.

3 import assert from 'node:assert'
                     ~~~~~~~~~~~~~

src/throttle.test.mts:7:11 - error TS2580: Cannot find name 'Buffer'. Do you need to install type definitions for node? Try `npm i --save-dev @types/node`.

7     yield Buffer.allocUnsafe(chunkSize)
            ~~~~~~

src/timeout.mts:1:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.

1 import assert from 'node:assert'
                     ~~~~~~~~~~~~~

src/timeout.test.mts:1:30 - error TS2307: Cannot find module 'node:test' or its corresponding type declarations.

1 import { describe, it } from 'node:test'
                               ~~~~~~~~~~~

src/timeout.test.mts:2:20 - error TS2307: Cannot find module 'node:assert' or its corresponding type declarations.

2 import assert from 'node:assert'
                     ~~~~~~~~~~~~~


Found 10 errors in 6 files.

Errors  Files
     1  src/synchronized.mts:1
     2  src/synchronized.test.mts:1
     1  src/throttle.mts:1
     3  src/throttle.test.mts:2
     1  src/timeout.mts:1
     2  src/timeout.test.mts:1
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

fbeauchamp opened this pull request in vatesfr/xen-orchestra

open feat(backups): use generator instead of streams for backup and replication #8432

Tristis Oris

i found time to play with cloud init, most of examples looks outdated or don't work dunno why.

hostname: {name} don't work, only hostname: {name}%. Also i don't find that macros at official doc.

with manage_etc_hosts: true it changed /etc/hosts 127.0.1.1 basename to 127.0.1.1 basename test%. Maybe package itself bug, maybe XO problem.

preserve_hostname: false looks not required, i don't see any difference.

even if not use any network config, it change netplan (don't need it with dhcp).

network:
  version: 2
  ethernets:
    enX0:
      dhcp4: true

to

network:
  version: 2
  ethernets:
    enX0:
      match:
        macaddress: "my_mac"
      dhcp4: true
      dhcp6: true
      set-name: "enX0"

to save default netplan, need to use something like

network:
  version: 1
  config:
      subnets:
      - type: dhcp4
      type: physical

can't make disk resize work, it looks like rocket science. And this is most important part for me.

resize_rootfs: true
growpart:
  mode: auto
  devices: ['/']
  ignore_growroot_disabled: false

I'm fine enough with manually tuned templates, 99% time don't need to change anything except name\disk. Other tasks require manual attention anyway or already covered with ansible. Would be nice to see tutorial for IQ<3.

Tristis Oris

@florent yep it works now.

Tristis Oris

@florent

yarn install v1.22.22
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning Pattern ["@intlify/shared@latest"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-@intlify-shared-11.1.2-d780bc8eb4e3c3ef8a3e45c421f0f5ecf59651f2-integrity/node_modules/@intlify/shared" as pattern ["@intlify/shared@11.1.2","@intlify/shared@11.1.2","@intlify/shared@11.1.2"]. This could result in non-deterministic behavior, skipping.
warning Pattern ["@intlify/shared@next"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-@intlify-shared-11.0.0-rc.1-52a67aa12fccd9303027b48ebf017b75b7350283-integrity/node_modules/@intlify/shared" as pattern ["@intlify/shared@11.0.0-rc.1"]. This could result in non-deterministic behavior, skipping.
warning bare-os@3.4.0: The engine "bare" appears to be invalid.
[4/5] Linking dependencies...
warning "@commitlint/cli > @commitlint/load > cosmiconfig-typescript-loader@6.1.0" has unmet peer dependency "@types/node@*".
warning "@commitlint/cli > @commitlint/load > cosmiconfig-typescript-loader@6.1.0" has unmet peer dependency "typescript@>=5".
warning "@typescript-eslint/eslint-plugin > ts-api-utils@1.4.3" has unmet peer dependency "typescript@>=4.2.0".
warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "pinia@^3.0.1".
warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "vue@~3.5.13".
warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "vue-i18n@^11.1.2".
warning " > @xen-orchestra/web-core@0.18.0" has unmet peer dependency "vue-router@^4.5.0".
warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @xen-orchestra/rest-api > inversify@6.2.2" has unmet peer dependency "reflect-metadata@~0.2.2".
warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @xen-orchestra/rest-api > swagger-ui-express@5.0.1" has unmet peer dependency "express@>=4.0.0 || >=5.0.0-beta".
warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @vates/event-listeners-manager > tap > @tapjs/test > @isaacs/ts-node-temp-fork-for-pr-2009@10.9.7" has unmet peer dependency "@types/node@*".
warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @xen-orchestra/rest-api > inversify > @inversifyjs/core > @inversifyjs/reflect-metadata-utils@0.2.4" has unmet peer dependency "reflect-metadata@0.2.2".
warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @vates/event-listeners-manager > tap > @tapjs/asserts > tcompare > react-element-to-jsx-string@15.0.0" has unmet peer dependency "react@^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1 || ^18.0.0".
warning "workspace-aggregator-9e56f98d-aa3d-4c5b-aae2-45a491674f9d > @vates/event-listeners-manager > tap > @tapjs/asserts > tcompare > react-element-to-jsx-string@15.0.0" has unmet peer dependency "react-dom@^0.14.8 || ^15.0.1 || ^16.0.0 || ^17.0.1 || ^18.0.0".
warning Workspaces can only be enabled in private projects.
[5/5] Building fresh packages...
$ husky install
husky - Git hooks installed
Done in 39.86s.

but same error.

Tristis Oris

same error as above

@xen-orchestra/disk-transform:build: cache miss, executing 6b8daba7020cf148
@xen-orchestra/disk-transform:build: yarn run v1.22.22
@xen-orchestra/disk-transform:build: $ tsc
@xen-orchestra/disk-transform:build: src/SynchronizedDisk.mts(2,30): error TS2307: Cannot find module '@vates/generator-toolbox' or its corresponding type declarations.
@xen-orchestra/disk-transform:build: error Command failed with exit code 2.
@xen-orchestra/disk-transform:build: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
@xen-orchestra/disk-transform:build: ERROR: command finished with error: command (/opt/xo/xo-builds/xen-orchestra-202504071027/@xen-orchestra/disk-transform) /tmp/yarn--1744010914016-0.7668824784043369/yarn run build exited (2)
@xen-orchestra/disk-transform#build: command (/opt/xo/xo-builds/xen-orchestra-202504071027/@xen-orchestra/disk-transform) /tmp/yarn--1744010914016-0.7668824784043369/yarn run build exited (2)

 Tasks:    24 successful, 28 total
Cached:    0 cached, 28 total
  Time:    11.251s
Failed:    @xen-orchestra/disk-transform#build

 ERROR  run failed: command  exited (2)
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
+ rm -rf /opt/xo/xo-builds/xen-orchestra-202504071027

Tristis Oris

latest builds failed because of XO6. https://github.com/ronivay/XenOrchestraInstallerUpdater/issues/288

bluerockny created this issue in ronivay/XenOrchestraInstallerUpdater

closed Update failure #288

Tristis Oris

@olivierlambert but could it change old backup tasks\compability or something else? tests with real data obviously more helpful.

Tristis Oris

Could i test it on prod, along with real backups? or better to setup 2nd XO instance.

Tristis Oris

@olivierlambert haha, i'm watching you)

Tristis Oris

just got to another pc - it works, so looks like ISP problem. My bad.

Tristis Oris

you say here that this feature is implemented, but i don't see new menu. ec782