• 0 Votes
    114 Posts
    16k Views
    poddingueP
    Thanks a lot for your feeback, folks!
  • VDI export to VMDK results in a corrupted disk

    Solved Xen Orchestra
    13
    0 Votes
    13 Posts
    118 Views
    A
    So in the end, I think the best approach is to export the VHD and convert it, because the conversion has to be done anyway, and the native VHD export is faster.
  • 3 Votes
    4 Posts
    161 Views
    acebmxerA
    v0.8.0–v0.9.1 — multi-user accounts, self-update, built-in HTTPS, and date ranges. Three releases since the last update, so bundling them here. Date ranges. Collections, extractions, findings runs and the support package can now be scoped to a window instead of always covering the whole bundle — last 24h/7d/30d, since last reboot, or a custom range. XO's own routes still have no date filter, so the first download is unchanged size — the range only narrows what gets kept and reported afterwards. Redact on demand. A collection no longer has to redact immediately with whatever rules happen to be on at that moment. There's now a checkbox to store the raw bundle only, and a "Redact now" button on the card afterwards using whichever rules are switched on when you press it. Multiple user accounts, roles, and an activity log. This was on the "what is coming" list in the first post and it's in now. Any number of accounts, three roles — admin, operator, viewer (read-only, but can still download what's already stored) — and an Activity page logging logins, settings changes, and job runs. Optional 2FA. Per-user TOTP, off by default, each account turns it on for itself. QR code setup, backup codes. Self-update from the UI, opt-in and off by default. Checking needs nothing extra; applying an update needs the Docker socket mounted in explicitly, since that's effectively host root, so it's a separate deliberate step — uncomment the socket mount and group_add block in docker-compose.yml, and it needs its own .env file (just DOCKER_GID=..., next to docker-compose.yml, not the same file as xcp-pulse.env) or docker compose up -d fails with unable to find group. .env.example has the one-liner to generate it. Built-in HTTPS, no reverse proxy required. Bundles nginx into the image to terminate TLS — self-signed cert on first start, or upload your own from Settings. A reverse proxy still works fine too. A user manual in the app itself, under /help — no need to leave XCP Pulse or check out the repo to read how something works. Fixed The account-requirements note from the first two posts was wrong in a way I only found by testing it properly: export:logs — the privilege log downloads actually need — isn't in any built-in XO role, but it can be granted through a custom role. "Test connection" was reading a catalogue that can't tell you that, and told every restricted account it needed full admin regardless. It now actually probes the download endpoint, and the docs walk through creating the custom role via three REST API calls (no UI for it yet on either XO version). A real concurrency bug: the shared SQLite connection wasn't locked across fetches, only execute, so two requests landing close together could crash a page reading jobs — reproduced it under the test suite's own concurrency test. Base-OS CVEs patched in the image build (perl-base, libc6, others); CI now runs a Trivy scan on every push. Still not fixed: the truncated-download-behind-a-reverse-proxy issue from the last post. Haven't found the actual cause yet. Also looking for anyone who can test against a remote proxy in a lab setup. Also open to any other suggestions, features, improvements, UI changes, etc... If any chance someone on Vates would test on their own time. Things like the support bundle and or the logs themselves are not being manipulated in any unwanted ways (for Vates or the project) https://github.com/acebmxer/xcp_pulse
  • MS-01 performance issues w/ Intel 226 NICs

    Hardware
    11
    0 Votes
    11 Posts
    3k Views
    B
    @Andrew said: pcie_aspm=disable Just to help anyone who would run into it. Disabling ASPM via dom0 settings/kernel did not resolve the issue. Had to disable it in BIOS (NUC13) After kernel level disable it did show: lspci -vv -s 55:00.0 | grep -E 'LnkCap|LnkCtl|LnkSta' LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L0s <2us, L1 <4us LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk+ LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Only after BIOS disable it showed lspci -vv -s 55:00.0 | grep -E 'LnkCap|LnkCtl|LnkSta' LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L1, Exit Latency L0s <2us, L1 <4us LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- Running on NUC13ANBi7 Hope it helps anyone running into this problem.
  • XO NFS option sec=krb5p encrypted transport

    Xen Orchestra
    2
    0 Votes
    2 Posts
    51 Views
    poddingueP
    I gave this a go on my lab XOA, and XO isn't touching your option at all: the failure quotes the command it ran, mount -o sec=krb5p -t nfs <server>:/path /run/xo-server/mounts/<remote-id>, and a control remote with no custom options mounted the same export fine. It does fail with mount.nfs: an incorrect mount option was specified, but that line tells you less than it looks like: I fed it sec=totalnonsense and got the identical message back, which is why it reads like XO refusing a valid option when it's really just relaying what mount.nfs said. The appliance ships rpc.gssd as part of nfs-common, so that part's there. What isn't there is /etc/krb5.keytab or /etc/krb5.conf, and the systemd unit carries ConditionPathExists=/etc/krb5.keytab, so the daemon never starts. Mine last failed that condition at boot eleven days ago and said nothing about it. I went one step further: dropping a keytab in place is enough for rpc.gssd to start and stay up, so that condition really is the only thing stopping it, and it doesn't need krb5.conf for that. Whether the mount then works needs a KDC and principals that agree with each other, and I couldn't get that far, so that part is still untested.
  • 0 Votes
    17 Posts
    1k Views
    M
    Hello @anthoineb, Following your suggestion to test a newer blktap release, we installed blktap-3.55.5-9.4.xcpng8.3.x86_64 and matching debuginfo on all three hypervisors. We then fully shut down and started all 21 OpenSearch data VMs, completing this on September 11. We verified that their running tapdisk executables matched the installed binary. Unfortunately, the stall recurred on September 13 on os-ott-data-1-5. For the affected tapdisk process (PID 2797568), we verified during the incident: /proc/2797568/exe pointed to /usr/libexec/tapdisk, without (deleted); its SHA-256 matched the installed executable; GDB loaded matching debug symbols, build ID dc98a78aff623c1cf12518663c859e3efa32413c. Before recovery: guest I/O made no progress; 255 requests were in flight and 256/256 tags were busy; I/O PSI full was approximately 96–97%, with four tasks in D state; tapdisk reported zero outstanding requests. The active td_xenblkif ring showed: req_prod = 866947066 req_cons = 866946810 rsp_prod = 866946810 rsp_prod_pvt = 866946810 nr_ents = 256 Thus, 256 requests were pending in the active PV ring but had not been consumed. After preserving the baseline GDB capture, our recovery controller executed your suggested call once: call (void)tapdisk_xenblkif_sched_chkrng(blkif) The call completed at 09:40:14 MSK (UTC+03:00). By 09:40:25, guest I/O was progressing again, inflight requests and busy tags were zero, and the local OpenSearch API was responding. The post-recovery GDB capture showed all four ring counters equal to 866967322. The node rejoined the cluster at 09:42:00 without restarting OpenSearch or rebooting the VM. So the same ring-processing stall still occurs with the running 9.4 binary, and the explicit ring-check call still restores I/O. We have complete before/after GDB captures and the recovery-call log available. Is there a newer build or a specific scheduler/event-channel state you would like us to capture during the next occurrence?
  • Error: Can't init vhd directory without using alias

    Unsolved Backup
    5
    1 Votes
    5 Posts
    93 Views
    A
    In any case, rolling back XO to an older build helps, so hopefully it will be fixed soon.
  • Master, commit a3139 failing backups

    Solved Backup
    21
    0 Votes
    21 Posts
    2k Views
    M
    I've just been hit with this also: Can't init vhd directory without using alias All S3 repo's running SeaweedFS, where fine the last time I used them (a few weeks ago). I don't use my xcpng servers on the homelab daily as I move to Unraid. Running Master, commit 99312
  • Date format on web interface: Only US format available?

    Unsolved Compute
    11
    2 Votes
    11 Posts
    1k Views
    J
    @julienXOvates said: @acomav We will actually propose to change the date and time format in XO6 settings, so you would be able to choose between : YYYY-MM-DD MM/DD/YYYY DD/MM/YYYY and 12h or 24h time format. Hopefully in one of the next 3 months ! I hope that will answer your need, otherwise let me know ! Thanks for putting this one the roadmap, for one of the next 3 months. That date order issue, has been bugging me as if you are rushed off your feet, may not remember that it’s the US date order. Also fits in more with the rest of European software used, or configured for the European localisation.
  • 0 Votes
    18 Posts
    2k Views
    J
    @johnnezero said: @john.c Thank you for all the great ideas, keep em' comming! Your welcome. If your other plugin was vibe coded it may be worth doing the same for the other. If you’re still working on it! @julienxovates In regards to the XO updates with the upcoming encrypted XO data store, by default (opt in for now). Is it ready for plugins like the one done by this author, and others in the future please?
  • 0 Votes
    4 Posts
    131 Views
    dthenotD
    @Dan Hello, So our investigation found that this change was introduced with commit https://github.com/xcp-ng/sm/commit/00637dd52e845d6016add9718fc9cc694aec9f0d It was aimed at EXTSR in particular, it appear that the first SR to be plugged is the one choosing the mode of sr-mount since util.makedirs also create the parent directory with the given mode. I have created a card for the issue on our side.
  • Backup remote repository structure

    Backup
    2
    0 Votes
    2 Posts
    67 Views
    P
    @abudef Hi, Do you think this doc can help you? Link to doc I think it needs some improvements, feel free to enhance it if you feel it is necessary!
  • VIDs are showing up as snapshot, but they are not.

    Unsolved XOSTOR
    7
    1
    0 Votes
    7 Posts
    447 Views
    K
    @olivierlambert I'm seeing a recurrence of the snapshot_of / hidden-disks issue after patching to xapi-26.1.16-1.2 (both hosts, rebooted). Unusual element this time: snapshot-fixer.py dry-run flags ~260 VDIs across the SR that all cite a single OpaqueRef which resolves to no VDI (xe vdi-list | grep <ref> returns nothing). Holding off on rewrite until I understand whether that single-dangling-ref pattern is safe, given the set appears to include base VDIs of legitimate snapshots. Full technical detail and questions posted on GitHub: https://github.com/xcp-ng/xcp/issues/844.
  • 0 Votes
    28 Posts
    5k Views
    K
    @poddingue I'm seeing a recurrence of the snapshot_of / hidden-disks issue after patching to xapi-26.1.16-1.2 (both hosts, rebooted). Unusual element this time: snapshot-fixer.py dry-run flags ~260 VDIs across the SR that all cite a single OpaqueRef which resolves to no VDI (xe vdi-list | grep <ref> returns nothing). Holding off on rewrite until I understand whether that single-dangling-ref pattern is safe, given the set appears to include base VDIs of legitimate snapshots. Full technical detail and questions posted on GitHub: https://github.com/xcp-ng/xcp/issues/844.
  • GPU Passthrough

    Management
    29
    3
    0 Votes
    29 Posts
    8k Views
    tjkreidlT
    @coolsport00 Sorry about the VMW need for the Cisco product. Sounds like you have a number of constraints, finances being I'm sure one of them! At least you have time on your hands and the means to experiment. You may, als, end up with a number of different platforms to meet your needs. We ran both Sun Microsystems and Red Hat Linux and Microsoft Windows servers, each taking on specific duties. It's far from ideal and probably not very cost-effective, but you do what you have to to get stuff to work.
  • 2 Votes
    4 Posts
    144 Views
    poddingueP
    Pretty cool, @samuelolavo, thanks for clarifying!
  • 2 Votes
    15 Posts
    1k Views
    acebmxerA
    I have received a very long update back from Veeam on my backup issues.... There appears to be another user with similar setup / issue not sure if that user is the OP this post specifically... Hello, Thank you for your patience. The QA team has finished the analysis, and I am going to outline the details as below: 1. CBT Inconsistency Issue Whenever you see the Warning about CBT showing: 2026-08-16 17:37:27.459 00079 ERROR | [XenRpcClient]: Failed ListChangedBlocks. Error: [Task 291af3d7-28c1-15a9-7f13-c6a1a12283e9 (Async.VDI.list_changed_blocks) failed: . SR_BACKEND_FAILURE_460. . Failed to calculate changed blocks for given VDIs. [opterr=Source and target VDI are unrelated]. We can go to the previous run and see that we get reports that the CBT of the previous snapshot is inconsistent and thus removed by XCP: 2026-08-16 09:54:34.120 00004 ERROR | [XenBackupManager]: Failed to retain the data for the snapshot 5a17fb31-2616-4e72-a85c-e235883c8c91 Veeam.Vbf.Common.Exceptions.ExceptionWithDetail: [Task 39c0f25f-e811-ee45-1b0b-e4a9ca9839ae (Async.VDI.data_destroy) failed: . VDI_NO_CBT_METADATA. OpaqueRef:a005d328-6da0-b8b7-34ef-0607ed44c2ed The team has been reviewing and testing, and this is what they see. We send the request for the snapshot to be created, and it is sent to the coordinator: xcp-ng-vyadytkn Aug 16 09:50:08 xcp-ng-vyadytkn SM: [1555017][MainThread] vdi_clone: introduced VDI: OpaqueRef:a005d328-6da0-b8b7-34ef-0607ed44c2ed (5a17fb31-2616-4e72-a85c-e235883c8c91) CBT shows as open and reading fine: Aug 16 09:50:08 xcp-ng-vyadytkn SM: [1555017][MainThread] ['/usr/sbin/cbt-util', 'set', '-n', '/var/run/sr-mount/7911c9c5-5f20-01e1-8b8d-39c6a98a2704/5a17fb31-2616-4e72-a85c-e235883c8c91.cbtlog', '-f', '1'] Aug 16 09:50:08 xcp-ng-vyadytkn SM: [1555017][MainThread] pread SUCCESS But look at how on xcp-ng-host2 the CBT gets marked as inconsistent by XCP, even though it's open and reading: Aug 16 09:51:05 xcp-ng-host2 SM: [1683689][MainThread] Changed Block Tracking metadata is inconsistent for disk 5a17fb31-2616-4e72-a85c-e235883c8c91. The Breakdown: xcp-ng-vyadytkn was the coordinator—the one we talk to, who then passes everything around to the hosts. xcp-ng-host2 was the host that the VM resided on at that time. xcp-ng-host2 is marking the CBT as inconsistent and deleting the snapshot CBT log, which means we cannot reference it on the next run. We do not see anything else interacting with the CBT besides that host. This matches exactly what we see with another client running the same setup. The team successfully replicated the environment, which is configured as follows: VM storage is NFS. The VM is running on a host that is not the coordinator. They have been able to reproduce this behavior occasionally, and the working theory is that the VM host keeps its own tracking separate from the coordinator. Part of the backup process requires the VM host to issue a pause/resume via a process called tapdisk. When it resumes, it pushes a data cache (likely inside the NFS cache), overwriting the CBT reference held by the coordinator server. It acts as a race condition—whoever pushes the CBT data last wins. If the VM host pushes last, it breaks what the coordinator is sending. Next Steps for CBT: The team is working to raise this issue directly with Vates so they can address the race condition. We ask that you also open a Vates ticket if possible to help draw more attention to the bug. We are trying to find ways to code around this race condition in the future, but there are currently no ETAs or guarantees. 2. Synthetic Full Failures (Delilah_ArcFS01) In addition to the CBT bug, the team discovered a separate issue. Recently, the synthetic fulls for Delilah_ArcFS01 have been failing. This appears to be related to the NFS repository: [30.08.2026 00:43:22.856] <24> [0007] Error (1) Failed to execute full transform task [30.08.2026 00:43:22.856] <24> [0007] Error (1) Agent: Failed to process method {Transform.CompileFIB}: NfsFileEx was already stopped. File: [Host:, Mount: [/volume1/veeam], Disk: [Delilah ArcFS01 Backup/Delilah ArcFS01 Backup_2026-08-29T232336.vib], Type: [nfs3 (1)]] (Veeam.Backup.Common.CCppComponentException) [30.08.2026 00:43:22.856] <24> [0007] Error (1) in c++: Failed to execute command Command: READ, Offset: 2523136, Data size: 659456, Chunk size: 131072 [30.08.2026 00:43:22.856] <24> [0007] Error (1) in c++: Failed to read file: Offset: 2523136, Block size: 659456, File: Path: [Host:, Mount: [/volume1/veeam], Disk: [Delilah ArcFS01 Backup/Delilah ArcFS01 Backup_2026-08-29T232336.vib], Type: [nfs3 (1)]], Handle: [01000702080061030000000007fffd5f65840fb20000000000000000150061038a2f7c0b0400610379207c0b], Read chunk size: 131072, Write chunk size: 131072, Read only: true Because it continuously fails during the synthetic full, it eventually causes the snapshot to be lost when retries fail. The team will change this logic in a future update. Action Items for the NFS Issue: To help prevent these snapshot loss failures, could you please provide the logs from the repository NFS (192.168.20.91)? Export the logs from the Veeam server and select the repository host. Provide the results of running this command directly on the Repository host: journalctl --since "30 days ago" > journal_repo.log Temporary Workaround: If you can, please temporarily switch to active fulls instead of synthetic fulls to help stabilize the job. Please let me know if you have any questions, and if you are able to raise that ticket with Vates. They also just responded back with this statment... Regarding the second part of the last email with the noticed Synthetic full issue, I actually would like you to also make this registry entry on the Veeam server and keep synthetic fulls enabled to see if it helps with that issue: Path: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication Name: Nfs3CommandWaitTimeoutSec Type: DWORD Value (In Decimal): 86400
  • 🛰️ XO 6: dedicated thread for all your feedback!

    Pinned Xen Orchestra
    265
    7 Votes
    265 Posts
    123k Views
    acebmxerA
    SDN controller documentation link - page not found... XO from sources latest commit - 280c0 [image: 1789054561550-screenshot-2026-09-10-113453.png] [image: 1789054584537-screenshot-2026-09-10-113617.png]
  • 0 Votes
    10 Posts
    563 Views
    poddingueP
    Thanks for the feedback, @dvinni !
  • XCP-ng 8.3 updates announcements and testing

    Pinned News
    680
    1 Votes
    680 Posts
    631k Views
    B
    @gduperrey Installed on all pools. No issues so far.