v0.6.0 — log collection is in. This was the piece the first post said was next, and it works end to end now.
A new Collect page runs a per-host collection as a background job: it pulls logs.tgz and the XAPI audit trail from Xen Orchestra, keeps the raw copies, and produces a redacted copy of each. On my own XCP-ng 8.3 pool that came to 434 MiB for the bundle and 769 MiB for the audit trail, 2.3 GiB stored across the five files, with 2,542,805 values masked — almost all of them session tokens.
The download is streamed a megabyte at a time and never held in memory, and every chunk is a cancellation point, so Cancel actually stops a transfer rather than waiting it out. A failed or cancelled download deletes its partial file instead of leaving a half-written bundle sitting there looking like a collection.
Both copies are kept and marked redacted — safe to send or raw — unmasked. Redaction is lossy, and the raw copy is the only thing that can answer "what did that used to say?" afterwards. Each collection writes the same redaction report the preview page produces, so you can see which rules fired — and which were switched off — before anything leaves the box. In the screenshot I had IPv4, MAC, UUIDs and hostnames turned off, and they show as off rather than as zero hits. That is deliberate: a partly-masked bundle should be obvious before you attach it to a ticket.
There is also a retention policy. The newest n collections are kept whatever their age, and only what is left is judged on age — a long gap in collecting can't empty the store. It shows you what a cleanup would delete and how much space that returns before you press the button.
Expect bugs. This is still very much a work in progress and it has only ever run against my own pool. If you try it in a different environment, assume things will break — please tell me what did. Known issues right now:
The red "This connection uses a restricted account" warning on the Collect page shows for every connection, including an administrator one. It is a wrong condition in the template, not a real permission check — ignore it if your account is admin. The screenshot below has it, over a collection that succeeded.
The Collect page says to expect "433 MB and 100 seconds per host". The 100 seconds is the download only; a full collection also fetches the audit trail and redacts both copies, which took 203 seconds on my pool. Budget three to four minutes.
Two more things worth flagging if you try this.
logs.tgz sends no Content-Length, so a reverse proxy that buffers will happily hand you a truncated archive with HTTP 200. Mine did. It now detects that and reports the bundle as cut short rather than as a protocol error, and repacks whatever did arrive instead of throwing the lot away. If you are behind Nginx Proxy Manager, proxy_buffering off; and proxy_max_temp_file_size 0; are what fixed the stall for me.
The account requirement from the first post has not changed — the log download needs export:logs on host, and on the instance I measured that is only reachable via Administrator. Inventory still works fine on Read only.
Next up: individual log categories pulled out of the cached bundle (~35 MB instead of 434 MiB), date ranges, and then findings.
https://github.com/acebmxer/xcp_pulse
[image: 1788812029125-screenshot_20260907_160937.png]
