Audit plugin: forward audit records to syslog in real time (PR #10460)

ludalm13

Hi everyone,

I opened a PR on the audit plugin and would love some feedback, and ideally a review from the XO team: https://github.com/vatesfr/xen-orchestra/pull/10460

The need

We need to send XO audit records to our central syslog server, like the rest of our infrastructure logs. Today the only way I found is to poll audit.getRecords from a cron job, which:

adds a delay (up to the polling interval),
needs to track which records were already exported, otherwise records are duplicated or missed,
needs a long-lived token of a real admin user stored on the machine.

What the PR does

It adds a logRecords option to the audit plugin, disabled by default. When it is enabled, each new audit record is also written to the xo-server logs as a single JSON line (namespace xo:xo-server-audit:records, level info):

xo:xo-server-audit:records INFO {"data":{...},"event":"apiCall","previousId":"...","subject":{...},"time":...,"id":"..."}

The records then follow the existing log pipeline in real time, either stdout → journald → rsyslog or the built-in [logs.transport.syslog], with no extra credentials.

The record is logged after it has been stored, so the log line is exactly the stored record, including id and previousId.
Nothing changes when the option is disabled, and the hash chain is not affected.
I tested it on a from-sources install with rsyslog forwarding to a remote server.

Related: #10461 (already merged) fixed actions made from XO 5 (/v5) being recorded with 127.0.0.1 / ::1 as the user IP. With both changes, the forwarded records contain the real client IP.

Questions

Does this approach look acceptable to the XO team, or would you prefer something else (e.g. a dedicated syslog transport in the plugin)?
Is anyone else exporting audit logs to a SIEM/syslog, and would this fit your use case?

Thanks!

poddingue

For anyone reading along who needs the forwarding part today, the [logs.transport.syslog] setting this PR builds on is documented, though on the backup settings page rather than next to the audit log, which isn't where I'd have looked: https://docs.xen-orchestra.com/backups-and-dr/backup-features-and-settings#send-xo-logs-to-an-external-syslog-server

On the design question (a log line through the existing pipeline versus a dedicated syslog transport in the plugin) I don't know the plugin well enough to have an opinion worth reading, so I think it's one for @Team-XO-Backend. The PR has been open since late September without a reviewer, so a look from them would help it along.

I'm also curious about your second question: if others here already ship audit records to a SIEM, saying so would show how many people need this.

dknight-bg

I believe that this functionality is very important for corporate clients with subscription like us.

Check also : https://feedback.vates.tech/posts/88/audit-log-forward-events-to-remote-syslog-record-successful-logins

poddingue

@dknight-bg the Feeder post you linked asks for two things: sending audit events to syslog, which is what this PR does, and recording successful logins, which as far as I can tell the PR doesn't touch.

If others reading this need either one, a vote on https://feedback.vates.tech/posts/88/audit-log-forward-events-to-remote-syslog-record-successful-logins is the easiest way to show it. The PR is still waiting for a reviewer, as far as I can see.

ludalm13

To add some context on our own use case: our standard xo-server logs already go to syslog through [logs.transport.syslog] and on to our central log server, and that works fine.

The problem is that a lot of the useful information only exists in the audit log: who did what, which API calls were made, from which IP. Right now that data stays inside XO and we can't correlate it with the rest of our infrastructure.

For us it is also a compliance issue. We are subject to PCI-DSS, and Requirement 10 expects administrative actions to be logged, promptly sent to a central log server and reviewed, ideally with automated tools. XO is where our virtualization admin actions happen, so its audit log is exactly the data an auditor wants to see in the SIEM.

Having the audit records go through the existing pipeline, as this PR does, would make life much easier: no polling script, no long-lived admin token, and everything ends up in our SIEM in real time. @Team-XO-Backend, a review would be much appreciated.