libfuse.so: error adding symbols: File in wrong format
-
@maverick the arm library should already have been loaded . My guess is that either the platform or the os aren't referenced in https://github.com/fuse-friends/fuse-shared-library/blob/master/index.js which is used by fuse native . Maybe a arm64 ?
-
@florent I'm not sure about your comment when you say "the arm library should already have been loaded" you mean in XO or on the system?
the platform reference here isaarch64. The fuse library for arm on the system is loaded, but what it seems to me from the log output is that the build process calls for/opt/xen-orchestra/node_modules/fuse-native/buildwhich in turn references/opt/xen-orchestra/node_modules/fuse-shared-library-linux/libfuse/lib/libfuse.soexplicitly. Again I'm not a developer so idk, but couldn't be a way for this to rely on the systems fuse-libs instead of those added tonode_modules?In the meanwhile I did the following: changed to the branch mentioned and re-did the process. Not sure if should have gone with different commands to apply this solution, but here's the output:
# git checkout fix_fuse_dependancy_arm Branch 'fix_fuse_dependancy_arm' set up to track remote branch 'fix_fuse_dependancy_arm' from 'origin'. Switched to a new branch 'fix_fuse_dependancy_arm' [root@xo-admin xen-orchestra]# git checkout . Updated 0 paths from the index [root@xo-admin xen-orchestra]# git pull --ff-only Already up to date. [root@xo-admin xen-orchestra]# yarn yarn install v1.22.19 warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json. [1/5] Validating package.json... [2/5] Resolving packages... [3/5] Fetching packages... [4/5] Linking dependencies... warning "workspace-aggregator-50b94db7-531c-4ce5-9ccb-8eb11c5cad3f > @xen-orchestra/fs > @aws-sdk/lib-storage@3.171.0" has unmet peer dependency "@aws-sdk/abort-controller@^3.0.0". warning Workspaces can only be enabled in private projects. [5/5] Building fresh packages... [1/12] ⠈ husky [6/12] ⠈ argon2 [8/12] ⠈ vuepress [7/12] ⠈ leveldown error /opt/xen-orchestra/node_modules/fuse-native: Command failed. Exit code: 1 Command: node-gyp-build Arguments: Directory: /opt/xen-orchestra/node_modules/fuse-native Output: gyp info it worked if it ends with ok gyp info using node-gyp@8.2.0 gyp info using node@16.13.0 | linux | arm64 gyp info find Python using Python version 3.6.8 found at "/usr/bin/python3" gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/opt/xen-orchestra/node_modules/fuse-native/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/root/.cache/node-gyp/16.13.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/16.13.0', gyp info spawn args '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/16.13.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/opt/xen-orchestra/node_modules/fuse-native', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp info spawn make gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ] make: Entering directory '/opt/xen-orchestra/node_modules/fuse-native/build' CC(target) Release/obj.target/fuse/fuse-native.o /opt/xen-orchestra/node_modules/fuse-shared-library-linux/libfuse/lib/libfuse.so: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status make: *** [fuse.target.mk:146: Release/obj.target/fuse.node] Error 1 gyp ERR! build error gyp ERR! stack Error: `make` failed with exit code: 2 gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23) gyp ERR! stack at ChildProcess.emit (node:events:390:28) gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) gyp ERR! System Linux 5.4.17-2136.307.3.1.el8uek.aarch64 gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" gyp ERR! cwd /opt/xen-orchestra/node_modules/fuse-native gyp ERR! node -v v16.13.0 gyp ERR! node-gyp -v v8.2.0 -
@maverick fuse-vhd is now an optionnal dependency it should still failed the build of this dependency, but should continue the global build
I am investigating and will keep you informed
-
@florent alright thank you so much! I got this topic with notifications so whenever you're ready for me to test just lmk I'll do it asap.
In the meanwhile, and just to scratch this off my mind, using the fuse-libraries that come with the system and are installed via rpm, is not viable?
-
@maverick using system li b will add a tight dependency, it will be harder to make it work on exotic systems
I pushed a new patch on the same branch, can you tell me if it solves the problem ?
-
@florent when you mention "exotic systems" what kind of systems are you thinking about? Seems that this Oracle Linux on ARM is also "exotic" no?
But wouldn't using the packages provided in the system repos solve the problem of said "exotic systems", since the packages provided are made for said systems, instead of risking using some libraries which supposedly are good for all, but in the end not really compatible with something which is the case here?Testing the new patch:
1st for context I had rolled back using git reset --hard commit and experiencing the issues I exposed on this topic:
https://xcp-ng.org/forum/topic/6347/xen-orchestra-broken-after-rollback/2nd:
# git checkout fix_fuse_dependancy_arm Switched to branch 'fix_fuse_dependancy_arm' Your branch is behind 'origin/fix_fuse_dependancy_arm' by 41 commits, and can be fast-forwarded. (use "git pull" to update your local branch) [root@xo-admin xen-orchestra]# git pull --ff-only remote: Enumerating objects: 9, done. remote: Counting objects: 100% (9/9), done. remote: Compressing objects: 100% (2/2), done. remote: Total 9 (delta 7), reused 9 (delta 7), pack-reused 0 Unpacking objects: 100% (9/9), 986 bytes | 140.00 KiB/s, done. From https://github.com/vatesfr/xen-orchestra dc571a581..dd0b71bca fix_fuse_dependancy_arm -> origin/fix_fuse_dependancy_arm Updating 37cdbc19e..dd0b71bca Fast-forward @vates/fuse-vhd/.npmignore | 1 + @vates/fuse-vhd/index.js | 66 ++ @vates/fuse-vhd/package.json | 32 + @xen-orchestra/backups-cli/package.json | 2 +- @xen-orchestra/backups/RemoteAdapter.js | 204 +++-- @xen-orchestra/backups/_cleanVm.js | 29 +- @xen-orchestra/backups/_deltaVm.js | 2 +- @xen-orchestra/backups/docs/VM backups/README.md | 14 +- @xen-orchestra/backups/package.json | 10 +- @xen-orchestra/backups/writers/DeltaBackupWriter.js | 38 +- @xen-orchestra/backups/writers/FullBackupWriter.js | 5 +- @xen-orchestra/backups/writers/_MixinBackupWriter.js | 43 +- @xen-orchestra/fs/package.json | 2 +- @xen-orchestra/mixins/package.json | 4 +- @xen-orchestra/proxy-cli/package.json | 2 +- @xen-orchestra/proxy/app/mixins/api.mjs | 32 +- @xen-orchestra/proxy/app/mixins/backups.mjs | 1 + @xen-orchestra/proxy/package.json | 12 +- @xen-orchestra/xapi/package.json | 2 +- CHANGELOG.md | 3961 +---------------------------------------------------------------------------------------- CHANGELOG.pre-2022.md | 3913 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ CHANGELOG.unreleased.md | 21 +- docs/configuration.md | 2 +- docs/users.md | 4 - package.json | 4 +- packages/vhd-cli/package.json | 6 +- packages/vhd-lib/Vhd/VhdAbstract.js | 34 + packages/vhd-lib/merge.js | 5 +- packages/vhd-lib/package.json | 8 +- packages/xen-api/examples/package.json | 2 +- packages/xen-api/package.json | 2 +- packages/xo-server-auth-saml/src/index.js | 1 + packages/xo-server-sdn-controller/package.json | 2 +- packages/xo-server-test/package.json | 4 +- packages/xo-server/config.toml | 11 + packages/xo-server/docs/file-restoration.md | 2 +- packages/xo-server/package.json | 14 +- packages/xo-server/src/api/vm.mjs | 1 + packages/xo-server/src/xapi-object-to-xo.mjs | 5 +- packages/xo-server/src/xapi/index.mjs | 24 +- packages/xo-server/src/xo-mixins/authentication.mjs | 2 +- packages/xo-server/src/xo-mixins/backups-remote-adapter.mjs | 2 + packages/xo-server/src/xo-mixins/plugins.mjs | 2 +- packages/xo-server/src/xo-mixins/remotes.mjs | 1 - packages/xo-vmdk-to-vhd/package.json | 2 +- packages/xo-web/package.json | 2 +- packages/xo-web/src/common/combobox.js | 5 +- packages/xo-web/src/common/intl/locales/es.js | 3 - packages/xo-web/src/common/intl/locales/fr.js | 4 - packages/xo-web/src/common/intl/locales/it.js | 3 - packages/xo-web/src/common/intl/locales/tr.js | 3 - packages/xo-web/src/common/intl/messages.js | 1 - packages/xo-web/src/common/json-schema-input/string-input.js | 1 + packages/xo-web/src/common/selectors.js | 14 +- packages/xo-web/src/xo-app/backup/file-restore/index.js | 6 +- packages/xo-web/src/xo-app/sr/tab-advanced.js | 2 +- packages/xo-web/src/xo-app/tasks/index.js | 34 +- yarn.lock | 7672 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------------------------------------------------------- 58 files changed, 8271 insertions(+), 8015 deletions(-) create mode 120000 @vates/fuse-vhd/.npmignore create mode 100644 @vates/fuse-vhd/index.js create mode 100644 @vates/fuse-vhd/package.json create mode 100644 CHANGELOG.pre-2022.md [root@xo-admin xen-orchestra]# yarn yarn install v1.22.19 warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json. [1/5] Validating package.json... [2/5] Resolving packages... [3/5] Fetching packages... [4/5] Linking dependencies... warning "workspace-aggregator-e6cb44f0-384c-413a-acbb-bf64e1e23946 > @xen-orchestra/fs > @aws-sdk/lib-storage@3.171.0" has unmet peer dependency "@aws-sdk/abort-controller@^3.0.0". warning Workspaces can only be enabled in private projects. [5/5] Building fresh packages... [-/2] ⡀ waiting... warning Error running install script for optional dependency: "/opt/xen-orchestra/node_modules/fuse-native: Command failed. Exit code: 1 Command: node-gyp-build Arguments: Directory: /opt/xen-orchestra/node_modules/fuse-native Output: gyp info it worked if it ends with ok gyp info using node-gyp@8.2.0 gyp info using node@16.13.0 | linux | arm64 gyp info find Python using Python version 3.6.8 found at \"/usr/bin/python3\" gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/opt/xen-orchestra/node_modules/fuse-native/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/root/.cache/node-gyp/16.13.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/16.13.0', gyp info spawn args '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/16.13.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/opt/xen-orchestra/node_modules/fuse-native', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp info spawn make gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ] /opt/xen-orchestra/node_modules/fuse-shared-library-linux/libfuse/lib/libfuse.so: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status make: *** [fuse.target.mk:146: Release/obj.target/fuse.node] Error 1 gyp ERR! build error gyp ERR! stack Error: `make` failed with exit code: 2 gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23) gyp ERR! stack at ChildProcess.emit (node:events:390:28) gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) gyp ERR! System Linux 5.4.17-2136.307.3.1.el8uek.aarch64 gyp ERR! command \"/usr/local/bin/node\" \"/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\" gyp ERR! cwd /opt/xen-orchestra/node_modules/fuse-native gyp ERR!make: Entering directory '/opt/xen-orchestra/node_modules/fuse-native/build' CC(target) Release/obj.target/fuse/fuse-native.o SOLINK_MODULE(target) Release/obj.target/fuse.node make: Leaving directory '/opt/xen-orchestra/node_modules/fuse-native/build' node -v v16.13.0 gyp ERR! node-gyp -v v8.2.0 gyp ERR! not ok" Done in 23.82s. -
@maverick I really want to focus on making it work in our system.
Can you try to runnpm installinstead ofyarn? -
@florent
Thank you very much I appreciate immensely your effort.Ok so doing
npm installinstead ofyarn, this is the output:# git checkout fix_fuse_dependancy_arm Already on 'fix_fuse_dependancy_arm' Your branch is behind 'origin/fix_fuse_dependancy_arm' by 107 commits, and can be fast-forwarded. (use "git pull" to update your local branch) [root@xo-admin xen-orchestra]# git pull --ff-only remote: Enumerating objects: 7, done. remote: Counting objects: 100% (7/7), done. remote: Total 7 (delta 6), reused 7 (delta 6), pack-reused 0 Unpacking objects: 100% (7/7), 697 bytes | 116.00 KiB/s, done. From https://github.com/vatesfr/xen-orchestra * [new branch] fix_more_retry_s3 -> origin/fix_more_retry_s3 Updating d87db05b2..dd0b71bca Fast-forward .github/ISSUE_TEMPLATE/feature_request.md | 1 - @vates/fuse-vhd/.npmignore | 1 + @vates/fuse-vhd/index.js | 66 +++ @vates/fuse-vhd/package.json | 32 ++ @xen-orchestra/backups-cli/package.json | 6 +- @xen-orchestra/backups/Backup.js | 2 +- @xen-orchestra/backups/RemoteAdapter.js | 204 ++++++--- @xen-orchestra/backups/Task.js | 6 +- @xen-orchestra/backups/_VmBackup.js | 37 +- @xen-orchestra/backups/_cleanVm.integ.spec.js | 50 +- @xen-orchestra/backups/_cleanVm.js | 100 ++-- @xen-orchestra/backups/_deltaVm.js | 2 +- @xen-orchestra/backups/_forkStreamUnpipe.js | 11 +- @xen-orchestra/backups/_isValidXva.js | 6 +- @xen-orchestra/backups/docs/VM backups/README.md | 14 +- @xen-orchestra/backups/package.json | 14 +- @xen-orchestra/backups/writers/DeltaBackupWriter.js | 38 +- @xen-orchestra/backups/writers/FullBackupWriter.js | 5 +- @xen-orchestra/backups/writers/_MixinBackupWriter.js | 43 +- @xen-orchestra/cr-seed-cli/package.json | 2 +- @xen-orchestra/fs/docs/encryption.md | 19 + @xen-orchestra/fs/package.json | 7 +- @xen-orchestra/fs/src/_encryptor.js | 71 +++ @xen-orchestra/fs/src/abstract.js | 168 +++++-- @xen-orchestra/fs/src/index.js | 5 +- @xen-orchestra/fs/src/local.js | 98 +++- @xen-orchestra/fs/src/s3.js | 8 + @xen-orchestra/fs/src/smb-mount.js | 23 - @xen-orchestra/fs/src/smb.js | 168 +------ @xen-orchestra/mixins/Hooks.mjs | 31 +- @xen-orchestra/mixins/SslCertificate.mjs | 4 +- @xen-orchestra/mixins/package.json | 6 +- @xen-orchestra/proxy-cli/package.json | 2 +- @xen-orchestra/proxy/app/mixins/api.mjs | 107 +++-- @xen-orchestra/proxy/app/mixins/backups.mjs | 1 + @xen-orchestra/proxy/package.json | 20 +- @xen-orchestra/upload-ova/package.json | 2 +- @xen-orchestra/xapi/package.json | 6 +- CHANGELOG.md | 3987 +++------------------------------------------------------------------------------------------------------------------------------------------------------------- CHANGELOG.pre-2022.md | 3913 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ CHANGELOG.unreleased.md | 18 + SECURITY.md | 10 +- docs/backup_troubleshooting.md | 3 + docs/configuration.md | 2 +- docs/sdn_controller.md | 2 +- docs/users.md | 4 - docs/xoa.md | 2 + package.json | 8 +- packages/complex-matcher/index.js | 2 +- packages/vhd-cli/commands/merge.js | 4 +- packages/vhd-cli/info.integ.spec.js | 12 +- packages/vhd-cli/package.json | 13 +- packages/vhd-lib/Vhd/VhdAbstract.js | 34 ++ packages/vhd-lib/Vhd/VhdDirectory.js | 9 +- packages/vhd-lib/Vhd/VhdFile.integ.spec.js | 16 +- packages/vhd-lib/Vhd/VhdFile.js | 1 + packages/vhd-lib/Vhd/VhdSynthetic.integ.spec.js | 9 +- packages/vhd-lib/aliases.js | 11 +- packages/vhd-lib/index.js | 2 - packages/vhd-lib/merge.integ.spec.js | 20 +- packages/vhd-lib/merge.js | 65 ++- packages/vhd-lib/openVhd.js | 3 +- packages/vhd-lib/package.json | 11 +- packages/xapi-explore-sr/package.json | 2 +- packages/xen-api/examples/package.json | 2 +- packages/xen-api/package.json | 4 +- packages/xen-api/src/index.js | 112 ++--- packages/xo-server-audit/package.json | 2 +- packages/xo-server-audit/src/index.js | 2 +- packages/xo-server-auth-saml/src/index.js | 3 +- packages/xo-server-sdn-controller/package.json | 2 +- packages/xo-server-test/package.json | 4 +- packages/xo-server-test/src/backupNg/__snapshots__/backupNg.spec.js.snap | 24 + packages/xo-server/config.toml | 21 +- packages/xo-server/docs/file-restoration.md | 2 +- packages/xo-server/package.json | 27 +- packages/xo-server/src/api/backup-ng.mjs | 7 +- packages/xo-server/src/api/disk.mjs | 2 +- packages/xo-server/src/api/host.mjs | 2 +- packages/xo-server/src/api/network.mjs | 8 +- packages/xo-server/src/api/pool.mjs | 32 -- packages/xo-server/src/api/sr.mjs | 10 +- packages/xo-server/src/api/vdi.mjs | 2 +- packages/xo-server/src/api/vm.mjs | 14 +- packages/xo-server/src/api/xosan.mjs | 5 +- packages/xo-server/src/collection/redis.mjs | 62 ++- packages/xo-server/src/db-cli.mjs | 59 ++- packages/xo-server/src/index.mjs | 14 +- packages/xo-server/src/recover-account-cli.mjs | 4 + packages/xo-server/src/sensitive-values.mjs | 2 +- packages/xo-server/src/xapi-object-to-xo.mjs | 4 + packages/xo-server/src/xapi/index.mjs | 28 +- packages/xo-server/src/xapi/mixins/storage.mjs | 49 +- packages/xo-server/src/xo-mixins/api.mjs | 146 ++++-- packages/xo-server/src/xo-mixins/authentication.mjs | 28 +- packages/xo-server/src/xo-mixins/backups-remote-adapter.mjs | 2 + packages/xo-server/src/xo-mixins/plugins.mjs | 2 +- packages/xo-server/src/xo-mixins/remotes.mjs | 1 - packages/xo-server/src/xo.mjs | 11 +- packages/xo-vmdk-to-vhd/package.json | 4 +- packages/xo-vmdk-to-vhd/src/ova-generate.integ.spec.js | 4 +- packages/xo-web/package.json | 4 +- packages/xo-web/src/common/combobox.js | 5 +- packages/xo-web/src/common/intl/locales/es.js | 3 - packages/xo-web/src/common/intl/locales/fr.js | 4 - packages/xo-web/src/common/intl/locales/it.js | 13 +- packages/xo-web/src/common/intl/locales/tr.js | 13 +- packages/xo-web/src/common/intl/messages.js | 25 +- packages/xo-web/src/common/json-schema-input/string-input.js | 1 + packages/xo-web/src/common/select-license.js | 61 +++ packages/xo-web/src/common/selectors.js | 14 +- packages/xo-web/src/common/xo/index.js | 33 +- packages/xo-web/src/xo-app/backup/file-restore/index.js | 6 +- packages/xo-web/src/xo-app/backup/restore/index.js | 2 +- packages/xo-web/src/xo-app/dashboard/health/unhealthyVdis.js | 42 +- packages/xo-web/src/xo-app/home/sr-item.js | 12 +- packages/xo-web/src/xo-app/jobs/new/index.js | 2 +- packages/xo-web/src/xo-app/logs/backup-ng/log-alert-body.js | 3 +- packages/xo-web/src/xo-app/proxies/index.js | 58 ++- packages/xo-web/src/xo-app/settings/remotes/remote.js | 33 ++ packages/xo-web/src/xo-app/sr/tab-advanced.js | 8 +- packages/xo-web/src/xo-app/tasks/index.js | 34 +- packages/xo-web/src/xo-app/xoa/licenses/index.js | 9 +- packages/xo-web/src/xo-app/xoa/licenses/proxies.js | 114 +++++ packages/xo-web/src/xo-app/xoa/licenses/xosan.js | 100 ++-- packages/xo-web/src/xo-app/xosan/index.js | 14 +- scripts/babel-eslintrc.js | 5 + yarn.lock | 3922 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------------------------- 128 files changed, 7921 insertions(+), 6875 deletions(-) create mode 120000 @vates/fuse-vhd/.npmignore create mode 100644 @vates/fuse-vhd/index.js create mode 100644 @vates/fuse-vhd/package.json create mode 100644 @xen-orchestra/fs/docs/encryption.md create mode 100644 @xen-orchestra/fs/src/_encryptor.js delete mode 100644 @xen-orchestra/fs/src/smb-mount.js create mode 100644 CHANGELOG.pre-2022.md create mode 100644 packages/xo-web/src/common/select-license.js create mode 100644 packages/xo-web/src/xo-app/xoa/licenses/proxies.js [root@xo-admin xen-orchestra]# npm install npm notice npm notice New minor version of npm available! 8.1.0 -> 8.19.2 npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.19.2 npm notice Run npm install -g npm@8.19.2 to update! npm notice npm ERR! code ERESOLVE npm ERR! ERESOLVE could not resolve npm ERR! npm ERR! While resolving: xo-web@5.103.0 npm ERR! Found: react-addons-test-utils@15.3.2 npm ERR! packages/xo-web/node_modules/react-addons-test-utils npm ERR! npm ERR! Could not resolve dependency: npm ERR! dev react-addons-test-utils@"^15.6.2" from xo-web@5.103.0 npm ERR! packages/xo-web npm ERR! xo-web@5.95.0 npm ERR! node_modules/xo-web npm ERR! workspace packages/xo-web from the root project npm ERR! npm ERR! Conflicting peer dependency: react-dom@15.7.0 npm ERR! node_modules/react-dom npm ERR! peer react-dom@"^15.4.2" from react-addons-test-utils@15.6.2 npm ERR! node_modules/react-addons-test-utils npm ERR! dev react-addons-test-utils@"^15.6.2" from xo-web@5.103.0 npm ERR! packages/xo-web npm ERR! xo-web@5.95.0 npm ERR! node_modules/xo-web npm ERR! workspace packages/xo-web from the root project npm ERR! npm ERR! Fix the upstream dependency conflict, or retry npm ERR! this command with --force, or --legacy-peer-deps npm ERR! to accept an incorrect (and potentially broken) dependency resolution. npm ERR! npm ERR! See /root/.npm/eresolve-report.txt for a full report. npm ERR! A complete log of this run can be found in: npm ERR! /root/.npm/_logs/2022-09-22T14_31_09_031Z-debug.log -
@maverick We are unable to reproduce this on our side, even with
npm@8.1.0. Which version of Node are you using?Can you retry with
npm@8.15.0? -
@julien-f Hi
Well I guess I'm using 8.1.0 as shown in the above log - i never cared to update npm as it was working:npm notice npm notice New minor version of npm available! 8.1.0 -> 8.19.2 npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.19.2I updated
npmto8.15.0as requested and here is the output:[root@xo-admin xen-orchestra]# npm install -g npm@8.15.0 removed 27 packages, changed 94 packages, and audited 202 packages in 3s 11 packages are looking for funding run `npm fund` for details found 0 vulnerabilities [root@xo-admin xen-orchestra]# git branch * fix_fuse_dependancy_arm master [root@xo-admin xen-orchestra]# git pull --ff-only Updating d87db05b2..dd0b71bca Fast-forward .github/ISSUE_TEMPLATE/feature_request.md | 1 - @vates/fuse-vhd/.npmignore | 1 + @vates/fuse-vhd/index.js | 66 +++ @vates/fuse-vhd/package.json | 32 ++ @xen-orchestra/backups-cli/package.json | 6 +- @xen-orchestra/backups/Backup.js | 2 +- @xen-orchestra/backups/RemoteAdapter.js | 204 ++++++--- @xen-orchestra/backups/Task.js | 6 +- @xen-orchestra/backups/_VmBackup.js | 37 +- @xen-orchestra/backups/_cleanVm.integ.spec.js | 50 +- @xen-orchestra/backups/_cleanVm.js | 100 ++-- @xen-orchestra/backups/_deltaVm.js | 2 +- @xen-orchestra/backups/_forkStreamUnpipe.js | 11 +- @xen-orchestra/backups/_isValidXva.js | 6 +- @xen-orchestra/backups/docs/VM backups/README.md | 14 +- @xen-orchestra/backups/package.json | 14 +- @xen-orchestra/backups/writers/DeltaBackupWriter.js | 38 +- @xen-orchestra/backups/writers/FullBackupWriter.js | 5 +- @xen-orchestra/backups/writers/_MixinBackupWriter.js | 43 +- @xen-orchestra/cr-seed-cli/package.json | 2 +- @xen-orchestra/fs/docs/encryption.md | 19 + @xen-orchestra/fs/package.json | 7 +- @xen-orchestra/fs/src/_encryptor.js | 71 +++ @xen-orchestra/fs/src/abstract.js | 168 +++++-- @xen-orchestra/fs/src/index.js | 5 +- @xen-orchestra/fs/src/local.js | 98 +++- @xen-orchestra/fs/src/s3.js | 8 + @xen-orchestra/fs/src/smb-mount.js | 23 - @xen-orchestra/fs/src/smb.js | 168 +------ @xen-orchestra/mixins/Hooks.mjs | 31 +- @xen-orchestra/mixins/SslCertificate.mjs | 4 +- @xen-orchestra/mixins/package.json | 6 +- @xen-orchestra/proxy-cli/package.json | 2 +- @xen-orchestra/proxy/app/mixins/api.mjs | 107 +++-- @xen-orchestra/proxy/app/mixins/backups.mjs | 1 + @xen-orchestra/proxy/package.json | 20 +- @xen-orchestra/upload-ova/package.json | 2 +- @xen-orchestra/xapi/package.json | 6 +- CHANGELOG.md | 3987 +++------------------------------------------------------------------------------------------------------------------------------------------------------------- CHANGELOG.pre-2022.md | 3913 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ CHANGELOG.unreleased.md | 18 + SECURITY.md | 10 +- docs/backup_troubleshooting.md | 3 + docs/configuration.md | 2 +- docs/sdn_controller.md | 2 +- docs/users.md | 4 - docs/xoa.md | 2 + package.json | 8 +- packages/complex-matcher/index.js | 2 +- packages/vhd-cli/commands/merge.js | 4 +- packages/vhd-cli/info.integ.spec.js | 12 +- packages/vhd-cli/package.json | 13 +- packages/vhd-lib/Vhd/VhdAbstract.js | 34 ++ packages/vhd-lib/Vhd/VhdDirectory.js | 9 +- packages/vhd-lib/Vhd/VhdFile.integ.spec.js | 16 +- packages/vhd-lib/Vhd/VhdFile.js | 1 + packages/vhd-lib/Vhd/VhdSynthetic.integ.spec.js | 9 +- packages/vhd-lib/aliases.js | 11 +- packages/vhd-lib/index.js | 2 - packages/vhd-lib/merge.integ.spec.js | 20 +- packages/vhd-lib/merge.js | 65 ++- packages/vhd-lib/openVhd.js | 3 +- packages/vhd-lib/package.json | 11 +- packages/xapi-explore-sr/package.json | 2 +- packages/xen-api/examples/package.json | 2 +- packages/xen-api/package.json | 4 +- packages/xen-api/src/index.js | 112 ++--- packages/xo-server-audit/package.json | 2 +- packages/xo-server-audit/src/index.js | 2 +- packages/xo-server-auth-saml/src/index.js | 3 +- packages/xo-server-sdn-controller/package.json | 2 +- packages/xo-server-test/package.json | 4 +- packages/xo-server-test/src/backupNg/__snapshots__/backupNg.spec.js.snap | 24 + packages/xo-server/config.toml | 21 +- packages/xo-server/docs/file-restoration.md | 2 +- packages/xo-server/package.json | 27 +- packages/xo-server/src/api/backup-ng.mjs | 7 +- packages/xo-server/src/api/disk.mjs | 2 +- packages/xo-server/src/api/host.mjs | 2 +- packages/xo-server/src/api/network.mjs | 8 +- packages/xo-server/src/api/pool.mjs | 32 -- packages/xo-server/src/api/sr.mjs | 10 +- packages/xo-server/src/api/vdi.mjs | 2 +- packages/xo-server/src/api/vm.mjs | 14 +- packages/xo-server/src/api/xosan.mjs | 5 +- packages/xo-server/src/collection/redis.mjs | 62 ++- packages/xo-server/src/db-cli.mjs | 59 ++- packages/xo-server/src/index.mjs | 14 +- packages/xo-server/src/recover-account-cli.mjs | 4 + packages/xo-server/src/sensitive-values.mjs | 2 +- packages/xo-server/src/xapi-object-to-xo.mjs | 4 + packages/xo-server/src/xapi/index.mjs | 28 +- packages/xo-server/src/xapi/mixins/storage.mjs | 49 +- packages/xo-server/src/xo-mixins/api.mjs | 146 ++++-- packages/xo-server/src/xo-mixins/authentication.mjs | 28 +- packages/xo-server/src/xo-mixins/backups-remote-adapter.mjs | 2 + packages/xo-server/src/xo-mixins/plugins.mjs | 2 +- packages/xo-server/src/xo-mixins/remotes.mjs | 1 - packages/xo-server/src/xo.mjs | 11 +- packages/xo-vmdk-to-vhd/package.json | 4 +- packages/xo-vmdk-to-vhd/src/ova-generate.integ.spec.js | 4 +- packages/xo-web/package.json | 4 +- packages/xo-web/src/common/combobox.js | 5 +- packages/xo-web/src/common/intl/locales/es.js | 3 - packages/xo-web/src/common/intl/locales/fr.js | 4 - packages/xo-web/src/common/intl/locales/it.js | 13 +- packages/xo-web/src/common/intl/locales/tr.js | 13 +- packages/xo-web/src/common/intl/messages.js | 25 +- packages/xo-web/src/common/json-schema-input/string-input.js | 1 + packages/xo-web/src/common/select-license.js | 61 +++ packages/xo-web/src/common/selectors.js | 14 +- packages/xo-web/src/common/xo/index.js | 33 +- packages/xo-web/src/xo-app/backup/file-restore/index.js | 6 +- packages/xo-web/src/xo-app/backup/restore/index.js | 2 +- packages/xo-web/src/xo-app/dashboard/health/unhealthyVdis.js | 42 +- packages/xo-web/src/xo-app/home/sr-item.js | 12 +- packages/xo-web/src/xo-app/jobs/new/index.js | 2 +- packages/xo-web/src/xo-app/logs/backup-ng/log-alert-body.js | 3 +- packages/xo-web/src/xo-app/proxies/index.js | 58 ++- packages/xo-web/src/xo-app/settings/remotes/remote.js | 33 ++ packages/xo-web/src/xo-app/sr/tab-advanced.js | 8 +- packages/xo-web/src/xo-app/tasks/index.js | 34 +- packages/xo-web/src/xo-app/xoa/licenses/index.js | 9 +- packages/xo-web/src/xo-app/xoa/licenses/proxies.js | 114 +++++ packages/xo-web/src/xo-app/xoa/licenses/xosan.js | 100 ++-- packages/xo-web/src/xo-app/xosan/index.js | 14 +- scripts/babel-eslintrc.js | 5 + yarn.lock | 3922 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------------------------- 128 files changed, 7921 insertions(+), 6875 deletions(-) create mode 120000 @vates/fuse-vhd/.npmignore create mode 100644 @vates/fuse-vhd/index.js create mode 100644 @vates/fuse-vhd/package.json create mode 100644 @xen-orchestra/fs/docs/encryption.md create mode 100644 @xen-orchestra/fs/src/_encryptor.js delete mode 100644 @xen-orchestra/fs/src/smb-mount.js create mode 100644 CHANGELOG.pre-2022.md create mode 100644 packages/xo-web/src/common/select-license.js create mode 100644 packages/xo-web/src/xo-app/xoa/licenses/proxies.js [root@xo-admin xen-orchestra]# npm install npm ERR! code ERESOLVE npm ERR! ERESOLVE could not resolve npm ERR! npm ERR! While resolving: xo-web@5.103.0 npm ERR! Found: react-addons-test-utils@15.3.2 npm ERR! packages/xo-web/node_modules/react-addons-test-utils npm ERR! npm ERR! Could not resolve dependency: npm ERR! dev react-addons-test-utils@"^15.6.2" from xo-web@5.103.0 npm ERR! packages/xo-web npm ERR! xo-web@5.95.0 npm ERR! node_modules/xo-web npm ERR! workspace packages/xo-web from the root project npm ERR! npm ERR! Conflicting peer dependency: react-dom@15.7.0 npm ERR! node_modules/react-dom npm ERR! peer react-dom@"^15.4.2" from react-addons-test-utils@15.6.2 npm ERR! node_modules/react-addons-test-utils npm ERR! dev react-addons-test-utils@"^15.6.2" from xo-web@5.103.0 npm ERR! packages/xo-web npm ERR! xo-web@5.95.0 npm ERR! node_modules/xo-web npm ERR! workspace packages/xo-web from the root project npm ERR! npm ERR! Fix the upstream dependency conflict, or retry npm ERR! this command with --force, or --legacy-peer-deps npm ERR! to accept an incorrect (and potentially broken) dependency resolution. npm ERR! npm ERR! See /root/.npm/eresolve-report.txt for a full report. npm ERR! A complete log of this run can be found in: npm ERR! /root/.npm/_logs/2022-09-23T10_33_52_057Z-debug-0.logPlease notice this machine is exclusive for XO. I never installed anything else than what is mentioned on the docs from source:
- Oracle Linux 8 minimal install;
- node;
- yarn;
- CERT forensics repo required on EL for libvhid-tools (wrongly named libvhdi-utils on the docs);
fetch the code with
git clone,
yarn
yarn buildconfig at
/etc/xo-server
forever-serviceto have xen orchestra always runningService redis has to be manually started and enabled or else it will fail.
systemctl enable --now redisall updates are done exclusively by running the command sequence:
git checkout . git pull --ff-only yarn yarn buildThe instance itself was running without issues for over an year. If you care to share all the updates you believe I should do to dependencies or wtv please just tell me which I should do and I'll do it.
-
@maverick I have no idea what you get this
npmerror, we haven't been able to reproduce.Have you tried running
npmwith--forceor--legacy-peer-depslike the error message suggests? -
@julien-f Hi Julien I haven't, but I will do it and let you know the results. I've already started another XO instance non-ARM until this one gets working again, let me know if you'd like me to wipe it clean and install from scratch.
Will do the--legacy-peer-depsand--forceafterwards depending on outcome -
It continues with errors.
# npm install --force npm WARN using --force Recommended protections disabled. up to date, audited 3281 packages in 10s 213 packages are looking for funding run `npm fund` for details 63 vulnerabilities (5 low, 8 moderate, 46 high, 4 critical) To address issues that do not require attention, run: npm audit fix To address all issues possible (including breaking changes), run: npm audit fix --force Some issues need review, and may require choosing a different dependency. Run `npm audit` for details. [root@xo-admin xen-orchestra]# yarn yarn install v1.22.19 warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json. [1/5] Validating package.json... [2/5] Resolving packages... warning vuepress > @vuepress/core > chokidar > fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2. warning Lockfile has incorrect entry for "styled-components@^3.1.5". Ignoring it. warning workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > gulp-watch > chokidar > fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2. [3/5] Fetching packages... [4/5] Linking dependencies... warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > @xen-orchestra/fs > @aws-sdk/lib-storage@3.72.0" has unmet peer dependency "@aws-sdk/abort-controller@^3.0.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > enzyme-adapter-react-15@1.4.3" has incorrect peer dependency "react@^15.5.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > enzyme-adapter-react-15@1.4.3" has incorrect peer dependency "react-dom@^15.5.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > enzyme-adapter-react-15@1.4.3" has incorrect peer dependency "react-test-renderer@^15.5.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-addons-test-utils@15.6.2" has incorrect peer dependency "react-dom@^15.4.2". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-dom@18.2.0" has incorrect peer dependency "react@^18.2.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-select@1.3.0" has incorrect peer dependency "react-dom@^0.14.9 || ^15.3.0 || ^16.0.0-rc || ^16.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-shortcuts@2.1.0" has incorrect peer dependency "react-dom@^0.14.8 || ^15 || ^16". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-test-renderer@18.2.0" has incorrect peer dependency "react@^18.2.0". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-virtualized@9.22.3" has incorrect peer dependency "react-dom@^15.3.0 || ^16.0.0-alpha". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > uncontrollable-input@0.1.1" has incorrect peer dependency "react@>=15.4.2". warning "workspace-aggregator-3a0c2cd3-09f2-49f2-8671-cef32d51497f > xo-web > react-test-renderer > react-shallow-renderer@16.15.0" has incorrect peer dependency "react@^16.0.0 || ^17.0.0 || ^18.0.0". warning Workspaces can only be enabled in private projects. [5/5] Building fresh packages... [9/11] ⡀ es5-ext [8/11] ⡀ vuepress [6/11] ⡀ argon2 [7/11] ⡀ leveldown warning Error running install script for optional dependency: "/opt/xen-orchestra/node_modules/fuse-native: Command failed. Exit code: 1 Command: node-gyp-build Arguments: Directory: /opt/xen-orchestra/node_modules/fuse-native Output: gyp info it worked if it ends with ok gyp info using node-gyp@9.0.0 gyp info using node@16.13.0 | linux | arm64 gyp info find Python using Python version 3.6.8 found at \"/usr/bin/python3\" gyp info spawn /usr/bin/python3 gyp info spawn args [ gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py', gyp info spawn args 'binding.gyp', gyp info spawn args '-f', gyp info spawn args 'make', gyp info spawn args '-I', gyp info spawn args '/opt/xen-orchestra/node_modules/fuse-native/build/config.gypi', gyp info spawn args '-I', gyp info spawn args '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi', gyp info spawn args '-I', gyp info spawn args '/root/.cache/node-gyp/16.13.0/include/node/common.gypi', gyp info spawn args '-Dlibrary=shared_library', gyp info spawn args '-Dvisibility=default', gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/16.13.0', gyp info spawn args '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp', gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/16.13.0/<(target_arch)/node.lib', gyp info spawn args '-Dmodule_root_dir=/opt/xen-orchestra/node_modules/fuse-native', gyp info spawn args '-Dnode_engine=v8', gyp info spawn args '--depth=.', gyp info spawn args '--no-parallel', gyp info spawn args '--generator-output', gyp info spawn args 'build', gyp info spawn args '-Goutput_dir=.' gyp info spawn args ] gyp info spawn make gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ] make: Entering directory '/opt/xen-orchestra/node_modules/fuse-native/build' CC(target) Release/obj.target/fuse/fuse-native.o SOLINK_MODULE(target) Release/obj.target/fuse.node make: Leaving directory '/opt/xen-orchestra/node_modules/fuse-native/build' /opt/xen-orchestra/node_modules/fuse-shared-library-linux/libfuse/lib/libfuse.so: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status make: *** [fuse.target.mk:146: Release/obj.target/fuse.node] Error 1 gyp ERR! build error gyp ERR! stack Error: `make` failed with exit code: 2 gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:194:23) gyp ERR! stack at ChildProcess.emit (node:events:390:28) gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12) gyp ERR! System Linux 5.4.17-2136.307.3.1.el8uek.aarch64 gyp ERR! command \"/usr/local/bin/node\" \"/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\" gyp ERR! cwd /opt/xen-orchestra/node_modules/fuse-native success Saved lockfile. Done in 64.63s. [root@xo-admin xen-orchestra]#Do you want me to erase this instance and do the install from source from scratch?
-
@maverick
- Do not run
yarn, it is replaced bynpm i - Start with
--legacy-peer-deps, only if it does not work try--force - If it still does not work, yes please, retry from scratch.
- Do not run
-
@julien-f hi there,
I'm sorry for my delay but was a bit overwhelmed with other tasks.
So running only
npm inow gives me the following output:# git checkout . Updated 13 paths from the index # git branch * fix_fuse_dependancy_arm master # git pull --ff-only remote: Enumerating objects: 79, done. remote: Counting objects: 100% (73/73), done. remote: Compressing objects: 100% (41/41), done. remote: Total 79 (delta 36), reused 61 (delta 30), pack-reused 6 Unpacking objects: 100% (79/79), 87.43 KiB | 3.01 MiB/s, done. From https://github.com/vatesfr/xen-orchestra f1ab62524..3d3b63a59 master -> origin/master + 2378c0882...2d9a44c6e tasks -> origin/tasks (forced update) Already up to date. # npm i npm WARN deprecated modular-cssify@12.1.3: Renamed to @modular-css/browserify npm WARN deprecated modular-css-core@12.1.3: Renamed to @modular-css/processor npm WARN deprecated chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies npm WARN deprecated core-js@2.6.12: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js. added 267 packages, removed 204 packages, changed 54 packages, and audited 3280 packages in 37s 209 packages are looking for funding run `npm fund` for details 70 vulnerabilities (5 low, 13 moderate, 49 high, 3 critical) To address issues that do not require attention, run: npm audit fix To address all issues possible (including breaking changes), run: npm audit fix --force Some issues need review, and may require choosing a different dependency. Run `npm audit` for details. #This seems to throw a lot of warnings but no fatal errors, do you confirm? Also it's not suggesting to use the
--legacy-peer-depsflag.After using these commands and rebooting the host, XO is stopped.
Isnpm ithe same asyarn; yarn build? Thank you!edit: when I get an hour I'll redo from scratch to see how that goes.
-
Hi, as I'm facing the same issue for the arm64 docker, I've build it in qemu with your
fix_fuse_dependancy_armbranch and it works for me. -
@maverick
npm ireplaceyarn, notyarn build. -
@julien-f so i should issue
yarn buildafter doingnpm igotcha. Give me a couple minutes will update you in an instant. -
after building when I open XO on the browser I only get an error saying
Cannot GET /I'll have to redo from source. Give me a couple of days please I've been a bit overwhelmed these last days
-
@maverick No hurries, we have a lot on our plates too.
