So in the end, I think the best approach is to export the VHD and convert it, because the conversion has to be done anyway, and the native VHD export is faster.

@coolsport00 Sorry about the VMW need for the Cisco product. Sounds like you have a number of constraints, finances being I'm sure one of them! At least you have time on your hands and the means to experiment. You may, als, end up with a number of different platforms to meet your needs. We ran both Sun Microsystems and Red Hat Linux and Microsoft Windows servers, each taking on specific duties. It's far from ideal and probably not very cost-effective, but you do what you have to to get stuff to work.

Thanks a lot for your feeback, folks!

@johnnezero said: @john.c Thank you for all the great ideas, keep em' comming! Your welcome. If your other plugin was vibe coded it may be worth doing the same for the other. If you’re still working on it! @julienxovates In regards to the XO updates with the upcoming encrypted XO data store, by default (opt in for now). Is it ready for plugins like the one done by this author, and others in the future please?

@florent & @poddingue I now have to heap snapshots. Could you provide somewhere to upload them, please? And I will do so as soon as I can.

@poddingue @Team-XO-Backend @pierrebrunet @florent I have the same issue in XO (delta backup to S3)... Caused by master commit adecd7306e2fa262ef77d71f83555d160f03aa03 { "data": { "mode": "delta", "reportWhen": "never" }, "id": "1789353842351", "jobId": "d6c0a656-62c5-4c39-a57a-f246b39f1cef", "jobName": "minio-test", "message": "backup", "scheduleId": "bd4ef436-fd85-4f16-bf9e-71d1d0c8586f", "start": 1789353842351, "status": "failure", "tasks": [ { "id": "0mu0n2une-6717eycsj3k", "start": 1789353844394, "status": "failure", "tasks": [ { "id": "0mu0n2unj-32t56n3u8vn", "start": 1789353844399, "status": "success", "end": 1789353844854, "result": { "removedFiles": [], "changedFiles": [], "merge": false, "size": 0 }, "message": "clean-vm" }, { "id": "0mu0n2vw5-jwlf68g7htp", "start": 1789353846005, "status": "success", "end": 1789353847486, "result": "6d4454f8-5735-0581-1eba-a66021f22a50", "message": "snapshot" }, { "id": "0mu0n2x1b-qze2zs5ytaq", "start": 1789353847487, "status": "failure", "tasks": [ { "id": "0mu0n2ylj-b5qu3cxprie", "start": 1789353849511, "status": "failure", "end": 1789353849570, "result": { "code": "NOT_SUPPORTED", "message": "Can't init vhd directory without using alias", "name": "Error", "stack": "Error: Can't init vhd directory without using alias\n at RemoteVhdDisk.init (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backup-archive/dist/disks/RemoteVhdDisk.mjs:74:41)\n at process.processTicksAndRejections (node:internal/process/task_queues:104:5)\n at async openDisposableDisk (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backup-archive/dist/disks/index.mjs:32:9)\n at async Promise.all (index 0)\n at async checkDisk (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/_runners/_writers/_checkDisk.mjs:5:3)\n at async DiskConsumerVhdDirectory.write (file:///opt/xo/xo-builds/xen-orchestra-202609132201/packages/vhd-lib/disk-consumer/DiskConsumerVhdDirectory.mjs:98:7)\n at async writeToVhdDirectory (file:///opt/xo/xo-builds/xen-orchestra-202609132201/packages/vhd-lib/disk-consumer/index.mjs:43:10)\n at async RemoteAdapter.writeVhd (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/RemoteAdapter.mjs:487:14)\n at async asyncEach.concurrency (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/_runners/_writers/IncrementalRemoteWriter.mjs:276:31)" }, "message": "transfer" } ], "end": 1789353849570, "result": { "code": "NOT_SUPPORTED", "message": "Can't init vhd directory without using alias", "name": "Error", "stack": "Error: Can't init vhd directory without using alias\n at RemoteVhdDisk.init (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backup-archive/dist/disks/RemoteVhdDisk.mjs:74:41)\n at process.processTicksAndRejections (node:internal/process/task_queues:104:5)\n at async openDisposableDisk (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backup-archive/dist/disks/index.mjs:32:9)\n at async Promise.all (index 0)\n at async checkDisk (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/_runners/_writers/_checkDisk.mjs:5:3)\n at async DiskConsumerVhdDirectory.write (file:///opt/xo/xo-builds/xen-orchestra-202609132201/packages/vhd-lib/disk-consumer/DiskConsumerVhdDirectory.mjs:98:7)\n at async writeToVhdDirectory (file:///opt/xo/xo-builds/xen-orchestra-202609132201/packages/vhd-lib/disk-consumer/index.mjs:43:10)\n at async RemoteAdapter.writeVhd (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/RemoteAdapter.mjs:487:14)\n at async asyncEach.concurrency (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/_runners/_writers/IncrementalRemoteWriter.mjs:276:31)" }, "message": "export", "data": { "id": "9890e0c4-ba3a-4810-8245-a49fdf16b16e", "isFull": false, "type": "remote" } } ], "infos": [ { "message": "Transfer data using NBD" }, { "message": "will delete snapshot data" }, { "data": { "vdiRef": "OpaqueRef:59ead5f8-710e-0e75-969e-bc59be6fab7f" }, "message": "Snapshot data has been deleted" } ], "end": 1789353852197, "result": { "code": "NOT_SUPPORTED", "message": "Can't init vhd directory without using alias", "name": "Error", "stack": "Error: Can't init vhd directory without using alias\n at RemoteVhdDisk.init (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backup-archive/dist/disks/RemoteVhdDisk.mjs:74:41)\n at process.processTicksAndRejections (node:internal/process/task_queues:104:5)\n at async openDisposableDisk (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backup-archive/dist/disks/index.mjs:32:9)\n at async Promise.all (index 0)\n at async checkDisk (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/_runners/_writers/_checkDisk.mjs:5:3)\n at async DiskConsumerVhdDirectory.write (file:///opt/xo/xo-builds/xen-orchestra-202609132201/packages/vhd-lib/disk-consumer/DiskConsumerVhdDirectory.mjs:98:7)\n at async writeToVhdDirectory (file:///opt/xo/xo-builds/xen-orchestra-202609132201/packages/vhd-lib/disk-consumer/index.mjs:43:10)\n at async RemoteAdapter.writeVhd (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/RemoteAdapter.mjs:487:14)\n at async asyncEach.concurrency (file:///opt/xo/xo-builds/xen-orchestra-202609132201/@xen-orchestra/backups/_runners/_writers/IncrementalRemoteWriter.mjs:276:31)" }, "message": "backup VM", "data": { "id": "a081f208-4e40-2ca5-c68e-86106af5a8ef", "type": "VM", "name_label": "Utest", "progress": 0 } } ], "end": 1789353852211, "infos": [ { "data": { "vms": [ "a081f208-4e40-2ca5-c68e-86106af5a8ef" ] }, "message": "vms" } ] }```

v0.8.0–v0.9.1 — multi-user accounts, self-update, built-in HTTPS, and date ranges. Three releases since the last update, so bundling them here. Date ranges. Collections, extractions, findings runs and the support package can now be scoped to a window instead of always covering the whole bundle — last 24h/7d/30d, since last reboot, or a custom range. XO's own routes still have no date filter, so the first download is unchanged size — the range only narrows what gets kept and reported afterwards. Redact on demand. A collection no longer has to redact immediately with whatever rules happen to be on at that moment. There's now a checkbox to store the raw bundle only, and a "Redact now" button on the card afterwards using whichever rules are switched on when you press it. Multiple user accounts, roles, and an activity log. This was on the "what is coming" list in the first post and it's in now. Any number of accounts, three roles — admin, operator, viewer (read-only, but can still download what's already stored) — and an Activity page logging logins, settings changes, and job runs. Optional 2FA. Per-user TOTP, off by default, each account turns it on for itself. QR code setup, backup codes. Self-update from the UI, opt-in and off by default. Checking needs nothing extra; applying an update needs the Docker socket mounted in explicitly, since that's effectively host root, so it's a separate deliberate step — uncomment the socket mount and group_add block in docker-compose.yml, and it needs its own .env file (just DOCKER_GID=..., next to docker-compose.yml, not the same file as xcp-pulse.env) or docker compose up -d fails with unable to find group. .env.example has the one-liner to generate it. Built-in HTTPS, no reverse proxy required. Bundles nginx into the image to terminate TLS — self-signed cert on first start, or upload your own from Settings. A reverse proxy still works fine too. A user manual in the app itself, under /help — no need to leave XCP Pulse or check out the repo to read how something works. Fixed The account-requirements note from the first two posts was wrong in a way I only found by testing it properly: export:logs — the privilege log downloads actually need — isn't in any built-in XO role, but it can be granted through a custom role. "Test connection" was reading a catalogue that can't tell you that, and told every restricted account it needed full admin regardless. It now actually probes the download endpoint, and the docs walk through creating the custom role via three REST API calls (no UI for it yet on either XO version). A real concurrency bug: the shared SQLite connection wasn't locked across fetches, only execute, so two requests landing close together could crash a page reading jobs — reproduced it under the test suite's own concurrency test. Base-OS CVEs patched in the image build (perl-base, libc6, others); CI now runs a Trivy scan on every push. Still not fixed: the truncated-download-behind-a-reverse-proxy issue from the last post. Haven't found the actual cause yet. Also looking for anyone who can test against a remote proxy in a lab setup. Also open to any other suggestions, features, improvements, UI changes, etc... If any chance someone on Vates would test on their own time. Things like the support bundle and or the logs themselves are not being manipulated in any unwanted ways (for Vates or the project) https://github.com/acebmxer/xcp_pulse

@Dan Hello, So our investigation found that this change was introduced with commit https://github.com/xcp-ng/sm/commit/00637dd52e845d6016add9718fc9cc694aec9f0d It was aimed at EXTSR in particular, it appear that the first SR to be plugged is the one choosing the mode of sr-mount since util.makedirs also create the parent directory with the given mode. I have created a card for the issue on our side.

Pretty cool, @samuelolavo, thanks for clarifying!

@poddingue Okay, we're back here. I was told I have a hung CIFS mount that no longer exists, so a host needs to reboot to clear it, and faulty XOSTOR resources. But I can't reboot the host until I can move VMs off. Is there any indication of how to repair the faulty XOSTOR resources so I can move the VMs and reboot the host? Thanks for taking a look.

I do have active directory setup in my lab environment. I just need to set aside some time to try it out between two domain joined hosts. Thanks for trying that! That's some good debugging.

We are fully aware of Broadcom's decision to stop providing the VDDK library to the cloud community or to reserve its download to some authorized partners and customers. As much as we regret this decision, we have to respect it. Because it impacts users who need to make backups, we are evaluating alternative options to address this situation. We will provide further guidance as this evaluation progresses but can't share more information for now.

@abudef Hi, Do you think this doc can help you? Link to doc I think it needs some improvements, feel free to enhance it if you feel it is necessary!

Thanks for the feedback, @dvinni !

@Andrew said: pcie_aspm=disable Just to help anyone who would run into it. Disabling ASPM via dom0 settings/kernel did not resolve the issue. Had to disable it in BIOS (NUC13) After kernel level disable it did show: lspci -vv -s 55:00.0 | grep -E 'LnkCap|LnkCtl|LnkSta' LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L0s <2us, L1 <4us LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk+ LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Only after BIOS disable it showed lspci -vv -s 55:00.0 | grep -E 'LnkCap|LnkCtl|LnkSta' LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L0s <2us, L1 <4us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- Running on NUC13ANBi7 Hope it helps anyone running into this problem.

Hello @anthoineb, Following your suggestion to test a newer blktap release, we installed blktap-3.55.5-9.4.xcpng8.3.x86_64 and matching debuginfo on all three hypervisors. We then fully shut down and started all 21 OpenSearch data VMs, completing this on September 11. We verified that their running tapdisk executables matched the installed binary. Unfortunately, the stall recurred on September 13 on os-ott-data-1-5. For the affected tapdisk process (PID 2797568), we verified during the incident: /proc/2797568/exe pointed to /usr/libexec/tapdisk, without (deleted); its SHA-256 matched the installed executable; GDB loaded matching debug symbols, build ID dc98a78aff623c1cf12518663c859e3efa32413c. Before recovery: guest I/O made no progress; 255 requests were in flight and 256/256 tags were busy; I/O PSI full was approximately 96–97%, with four tasks in D state; tapdisk reported zero outstanding requests. The active td_xenblkif ring showed: req_prod = 866947066 req_cons = 866946810 rsp_prod = 866946810 rsp_prod_pvt = 866946810 nr_ents = 256 Thus, 256 requests were pending in the active PV ring but had not been consumed. After preserving the baseline GDB capture, our recovery controller executed your suggested call once: call (void)tapdisk_xenblkif_sched_chkrng(blkif) The call completed at 09:40:14 MSK (UTC+03:00). By 09:40:25, guest I/O was progressing again, inflight requests and busy tags were zero, and the local OpenSearch API was responding. The post-recovery GDB capture showed all four ring counters equal to 866967322. The node rejoined the cluster at 09:42:00 without restarting OpenSearch or rebooting the VM. So the same ring-processing stall still occurs with the running 9.4 binary, and the explicit ring-check call still restores I/O. We have complete before/after GDB captures and the recovery-call log available. Is there a newer build or a specific scheduler/event-channel state you would like us to capture during the next occurrence?

I've just been hit with this also: Can't init vhd directory without using alias All S3 repo's running SeaweedFS, where fine the last time I used them (a few weeks ago). I don't use my xcpng servers on the homelab daily as I move to Unraid. Running Master, commit 99312

@julienXOvates said: @acomav We will actually propose to change the date and time format in XO6 settings, so you would be able to choose between : YYYY-MM-DD MM/DD/YYYY DD/MM/YYYY and 12h or 24h time format. Hopefully in one of the next 3 months ! I hope that will answer your need, otherwise let me know ! Thanks for putting this one the roadmap, for one of the next 3 months. That date order issue, has been bugging me as if you are rushed off your feet, may not remember that it’s the US date order. Also fits in more with the rest of European software used, or configured for the European localisation.