Remote syslog broken after update/reboot? - Changing it away, then back fixes.

[MajorP93]

@rzr No I did not customize anything related to rsyslog on XO / XCP-ng side. No tweaking of rsyslog config file(s) or similar.
I just setup a graylog server, enabled the syslog udp input there and configured rsyslog via Xen Orchestra like so:

I consider this the most basic setup.
Also: AFAIK there are no UI options for rsyslog custom rules. Enabling it and setting IP address/port is basically all that can be done. I read that Vates recommends to not fiddle with config files on dom0 and view XCP-ng as an appliance.
So maybe custom rsyslog config does not need to get considered as long as no new GUI features for that are planned.

[Forza]

I've been considering remote syslog too. Does enabling remote syslog remove local logging?

[stormi]

@MajorP93 Good to know that you have another pool to update. Then please also save the file prior to updating.

[rzr]

Did you set a system a custom locale settings ? please share ouput of locale command ? we are suspecting rsyslog issues

[MajorP93]

@stormi Yes, I am currently working on replicating the issue in our test environment. I will report back once done.

@rzr I configured locale via XCP-ng ISO installer. This is my locale output:

[14:38 xcpng01 ~]# locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

[MajorP93]

@stormi I was able to replicate this issue in our test environment.
Applying of updates --> rsyslog still working.
Rebooting after applying the updates --> rsyslog not working, same issue as previously explained.

I found out that rsyslog target IP address gets saved in /etc/rsyslog.d/xenserver.conf.
Here are the contents of the files in question:

Before applying the updates via yum:
/etc/rsyslog.conf

# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
#$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
#$OmitLocalLogging on

# File to store the position in the journal
#$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

/etc/rsyslog.d/xenserver.conf:

# Suppress duplicate messages and report "Last line repeated n times"
$RepeatedMsgReduction on

# Don't rate-limit messages - this isn't the right way to go about 
# reducing log size!
$IMUXSockRateLimitInterval 0
$SystemLogRateLimitInterval 0

# Ensure critical and higher level errors are logged synchronously.
$ActionFileEnableSync on
$outchannel crit_log,/var/log/crit.log,104857600,/etc/cron.daily/logrotate
*.crit;mail.none;authpriv.none;cron.none		:omfile:$crit_log
$ActionFileEnableSync off

# Log crit to the console as well
*.crit                                          /dev/hvc0

# Corosync logs useful things at warning
if $programname == 'corosync' and $syslogseverity <= 4 then /dev/hvc0

# HTTP disk server backend used by XHA (LINSTOR SR).
# Redirected to a specific log file instead of daemon.log (facility 3).
$outchannel xcp_http_disk_server_log,/var/log/xcp-http-nbd-server.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 3 and $programname == 'http-disk-server' then :omfile:$xcp_http_disk_server_log
& stop

# NBD server used by XHA (LINSTOR SR).
# Redirected to a specific log file instead of daemon.log (facility 3).
$outchannel xcp_nbd_http_server_log,/var/log/xcp-nbd-http-server.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 3 and $programname == 'nbd-http-server' then :omfile:$xcp_nbd_http_server_log
& stop

# Log in specific file when a DRBD log is matched.
# Redirected to a specific log file instead of kern.log (facility 0).
$outchannel drbd_kern_log,/var/log/drbd-kern.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 0 and re_match($msg, '^\\[[ ]*[0-9]+\\.[0-9]+\\] drbd([0-9]+)?:? ') then :omfile:$drbd_kern_log
& stop

# Log by facility.
$outchannel kern_log,/var/log/kern.log,104857600,/etc/cron.daily/logrotate
kern.*							:omfile:$kern_log

# dlm_controld logs to syslog local4
$outchannel daemon_log,/var/log/daemon.log,104857600,/etc/cron.daily/logrotate
daemon.*;local4.*					:omfile:$daemon_log

$outchannel user_log,/var/log/user.log,104857600,/etc/cron.daily/logrotate
user.*							:omfile:$user_log

# The authpriv file has restricted access.
$outchannel secure_log,/var/log/secure,104857600,/etc/cron.daily/logrotate
authpriv.*						:omfile:$secure_log

# Log all the mail messages in one place.
$outchannel mail_log,/var/log/maillog,104857600,/etc/cron.daily/logrotate
mail.*							:omfile:$mail_log

# Log cron stuff
$outchannel cron_log,/var/log/cron,104857600,/etc/cron.daily/logrotate
cron.*							:omfile:$cron_log

# Save boot messages also to boot.log
$outchannel boot_log,/var/log/boot.log,104857600,/etc/cron.daily/logrotate
local7.*						:omfile:$boot_log

# Xapi rbac audit log echoes to syslog local6
$outchannel audit_log,/var/log/audit.log,104857600,/etc/cron.daily/logrotate
local6.*						:omfile:$audit_log

# Xapi, xenopsd echo to syslog local5
$outchannel xensource_log,/var/log/xensource.log,104857600,/etc/cron.daily/logrotate
local5.*						:omfile:$xensource_log

# xenstore access to syslog local3
$outchannel xenstored_log,/var/log/xenstored-access.log,104857600,/etc/cron.daily/logrotate
local3.info						:omfile:$xenstored_log

# Storage Manager to syslog local2
$outchannel sm_log,/var/log/SMlog,104857600,/etc/cron.daily/logrotate
local2.info						:omfile:$sm_log

# Scheduled snapshots to syslog local1
$outchannel vmss_log,/var/log/VMSSlog,104857600,/etc/cron.daily/logrotate
local1.*						:omfile:$vmss_log

# xcp-rrdd-plugins (info and above) to local0
$outchannel xcp_rrdd_log,/var/log/xcp-rrdd-plugins.log,104857600,/etc/cron.daily/logrotate
local0.info						:omfile:$xcp_rrdd_log

# ignore default rules

*.* @10.10.160.27:5140
*.* ~

After applying the updates via yum:
/etc/rsyslog.conf:

# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
#$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
#$OmitLocalLogging on

# File to store the position in the journal
#$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

/etc/rsyslog.d/xenserver.conf:

# Suppress duplicate messages and report "Last line repeated n times"
$RepeatedMsgReduction on

# Don't rate-limit messages - this isn't the right way to go about 
# reducing log size!
$IMUXSockRateLimitInterval 0
$SystemLogRateLimitInterval 0

# Ensure critical and higher level errors are logged synchronously.
$ActionFileEnableSync on
$outchannel crit_log,/var/log/crit.log,104857600,/etc/cron.daily/logrotate
*.crit;mail.none;authpriv.none;cron.none		:omfile:$crit_log
$ActionFileEnableSync off

# Log crit to the console as well
*.crit                                          /dev/hvc0

# Corosync logs useful things at warning
if $programname == 'corosync' and $syslogseverity <= 4 then /dev/hvc0

# HTTP disk server backend used by XHA (LINSTOR SR).
# Redirected to a specific log file instead of daemon.log (facility 3).
$outchannel xcp_http_disk_server_log,/var/log/xcp-http-nbd-server.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 3 and $programname == 'http-disk-server' then :omfile:$xcp_http_disk_server_log
& stop

# NBD server used by XHA (LINSTOR SR).
# Redirected to a specific log file instead of daemon.log (facility 3).
$outchannel xcp_nbd_http_server_log,/var/log/xcp-nbd-http-server.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 3 and $programname == 'nbd-http-server' then :omfile:$xcp_nbd_http_server_log
& stop

# Log in specific file when a DRBD log is matched.
# Redirected to a specific log file instead of kern.log (facility 0).
$outchannel drbd_kern_log,/var/log/drbd-kern.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 0 and re_match($msg, '^\\[[ ]*[0-9]+\\.[0-9]+\\] drbd([0-9]+)?:? ') then :omfile:$drbd_kern_log
& stop

# Log by facility.
$outchannel kern_log,/var/log/kern.log,104857600,/etc/cron.daily/logrotate
kern.*							:omfile:$kern_log

# dlm_controld logs to syslog local4
$outchannel daemon_log,/var/log/daemon.log,104857600,/etc/cron.daily/logrotate
daemon.*;local4.*					:omfile:$daemon_log

$outchannel user_log,/var/log/user.log,104857600,/etc/cron.daily/logrotate
user.*							:omfile:$user_log

# The authpriv file has restricted access.
$outchannel secure_log,/var/log/secure,104857600,/etc/cron.daily/logrotate
authpriv.*						:omfile:$secure_log

# Log all the mail messages in one place.
$outchannel mail_log,/var/log/maillog,104857600,/etc/cron.daily/logrotate
mail.*							:omfile:$mail_log

# Log cron stuff
$outchannel cron_log,/var/log/cron,104857600,/etc/cron.daily/logrotate
cron.*							:omfile:$cron_log

# Save boot messages also to boot.log
$outchannel boot_log,/var/log/boot.log,104857600,/etc/cron.daily/logrotate
local7.*						:omfile:$boot_log

# Xapi rbac audit log echoes to syslog local6
$outchannel audit_log,/var/log/audit.log,104857600,/etc/cron.daily/logrotate
local6.*						:omfile:$audit_log

# Xapi, xenopsd echo to syslog local5
$outchannel xensource_log,/var/log/xensource.log,104857600,/etc/cron.daily/logrotate
local5.*						:omfile:$xensource_log

# xenstore access to syslog local3
$outchannel xenstored_log,/var/log/xenstored-access.log,104857600,/etc/cron.daily/logrotate
local3.info						:omfile:$xenstored_log

# Storage Manager to syslog local2
$outchannel sm_log,/var/log/SMlog,104857600,/etc/cron.daily/logrotate
local2.info						:omfile:$sm_log

# Scheduled snapshots to syslog local1
$outchannel vmss_log,/var/log/VMSSlog,104857600,/etc/cron.daily/logrotate
local1.*						:omfile:$vmss_log

# xcp-rrdd-plugins (info and above) to local0
$outchannel xcp_rrdd_log,/var/log/xcp-rrdd-plugins.log,104857600,/etc/cron.daily/logrotate
local0.info						:omfile:$xcp_rrdd_log

# ignore default rules
*.*							~

After rebooting:
/etc/rsyslog.conf:

# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
#$ModLoad imjournal # provides access to the systemd journal
$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
#$OmitLocalLogging on

# File to store the position in the journal
#$IMJournalStateFile imjournal.state


#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

/etc/rsyslog.d/xenserver.conf:

# Suppress duplicate messages and report "Last line repeated n times"
$RepeatedMsgReduction on

# Don't rate-limit messages - this isn't the right way to go about 
# reducing log size!
$IMUXSockRateLimitInterval 0
$SystemLogRateLimitInterval 0

# Ensure critical and higher level errors are logged synchronously.
$ActionFileEnableSync on
$outchannel crit_log,/var/log/crit.log,104857600,/etc/cron.daily/logrotate
*.crit;mail.none;authpriv.none;cron.none		:omfile:$crit_log
$ActionFileEnableSync off

# Log crit to the console as well
*.crit                                          /dev/hvc0

# Corosync logs useful things at warning
if $programname == 'corosync' and $syslogseverity <= 4 then /dev/hvc0

# HTTP disk server backend used by XHA (LINSTOR SR).
# Redirected to a specific log file instead of daemon.log (facility 3).
$outchannel xcp_http_disk_server_log,/var/log/xcp-http-nbd-server.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 3 and $programname == 'http-disk-server' then :omfile:$xcp_http_disk_server_log
& stop

# NBD server used by XHA (LINSTOR SR).
# Redirected to a specific log file instead of daemon.log (facility 3).
$outchannel xcp_nbd_http_server_log,/var/log/xcp-nbd-http-server.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 3 and $programname == 'nbd-http-server' then :omfile:$xcp_nbd_http_server_log
& stop

# Log in specific file when a DRBD log is matched.
# Redirected to a specific log file instead of kern.log (facility 0).
$outchannel drbd_kern_log,/var/log/drbd-kern.log,104857600,/etc/cron.daily/logrotate
if $syslogfacility == 0 and re_match($msg, '^\\[[ ]*[0-9]+\\.[0-9]+\\] drbd([0-9]+)?:? ') then :omfile:$drbd_kern_log
& stop

# Log by facility.
$outchannel kern_log,/var/log/kern.log,104857600,/etc/cron.daily/logrotate
kern.*							:omfile:$kern_log

# dlm_controld logs to syslog local4
$outchannel daemon_log,/var/log/daemon.log,104857600,/etc/cron.daily/logrotate
daemon.*;local4.*					:omfile:$daemon_log

$outchannel user_log,/var/log/user.log,104857600,/etc/cron.daily/logrotate
user.*							:omfile:$user_log

# The authpriv file has restricted access.
$outchannel secure_log,/var/log/secure,104857600,/etc/cron.daily/logrotate
authpriv.*						:omfile:$secure_log

# Log all the mail messages in one place.
$outchannel mail_log,/var/log/maillog,104857600,/etc/cron.daily/logrotate
mail.*							:omfile:$mail_log

# Log cron stuff
$outchannel cron_log,/var/log/cron,104857600,/etc/cron.daily/logrotate
cron.*							:omfile:$cron_log

# Save boot messages also to boot.log
$outchannel boot_log,/var/log/boot.log,104857600,/etc/cron.daily/logrotate
local7.*						:omfile:$boot_log

# Xapi rbac audit log echoes to syslog local6
$outchannel audit_log,/var/log/audit.log,104857600,/etc/cron.daily/logrotate
local6.*						:omfile:$audit_log

# Xapi, xenopsd echo to syslog local5
$outchannel xensource_log,/var/log/xensource.log,104857600,/etc/cron.daily/logrotate
local5.*						:omfile:$xensource_log

# xenstore access to syslog local3
$outchannel xenstored_log,/var/log/xenstored-access.log,104857600,/etc/cron.daily/logrotate
local3.info						:omfile:$xenstored_log

# Storage Manager to syslog local2
$outchannel sm_log,/var/log/SMlog,104857600,/etc/cron.daily/logrotate
local2.info						:omfile:$sm_log

# Scheduled snapshots to syslog local1
$outchannel vmss_log,/var/log/VMSSlog,104857600,/etc/cron.daily/logrotate
local1.*						:omfile:$vmss_log

# xcp-rrdd-plugins (info and above) to local0
$outchannel xcp_rrdd_log,/var/log/xcp-rrdd-plugins.log,104857600,/etc/cron.daily/logrotate
local0.info						:omfile:$xcp_rrdd_log

# ignore default rules
*.*							~

--> appearently after applying the updates my rsyslog target system got removed from /etc/rsyslog.d/xenserver.conf but this change got active after rebooting.

[stormi]

This post is deleted!

[stormi]

@MajorP93 Thanks. That clarifies what happened. I wrongly thought that it was managed in /etc/rsyslog.conf (which never gets overwritten), but it's in /etc/rsyslog.d/xenserver.conf, which can get overwritten if modified by an update.

[stormi]

The fix was already high in the priority list anyway, but I'll try to make sure we don't postpone it more. Had I realized sooner, I would have made it so we'd release it earlier.

According to the package definitions, we even overwrite it each time we update the xcp-ng-release-config package. I'm surprised that we haven't had more support requests.

I'll see if we can release a quick fix that just leaves the file alone until we apply the change that will move the remote configuration to its own file.

[olivierlambert]

There's a similar magic we've seen countless time with XO: you have a bug that nobody reports for years and suddenly it pops from various sources (or with many details so it's easy to spot), you think it's recently introduced, you check your diff and then after hours you realize it's there since a while

[MajorP93]

@stormi Thanks, that sounds great!
If there is something I can test / provide please let me know!

[rzr]

@MajorP93

Fix has been merged, expect a package in your updates soon.

Meanwhile check this notice about upcoming changes regarding remote syslog.

https://github.com/xcp-ng-rpms/xcp-ng-release/pull/41#issuecomment-3800419449

rzr opened this pull request in xcp-ng-rpms/xcp-ng-release

closed Preserve /etc/rsyslog.d/xenserver.conf if present #41

[michmoor0725]

This community is amazing Great job all around!

[MajorP93]

@rzr Thank you very much!

@michmoor0725 Absolutely! The community is another aspect of why working with XCP-ng is a lot more fun compared to working with VMWare!

[gduperrey]

The update should now be available. You can view the announcement here: https://xcp-ng.org/blog/2026/01/29/january-2026-security-and-maintenance-updates-for-xcp-ng-8-3-lts/

So don’t hesitate to update it and confirm that it properly addresses the issue.

[MajorP93]

@gduperrey Thanks!
Will test tomorrow as our internal lab / test environment is currently unavailable.
I will inform you about the results of my testing here.

Best regards

[MajorP93]

@gduperrey Hey,
I tested it at can confirm that after applying latest set of patches and rebooting remote syslog is still working fine.
It appears to be fixed, good job guys

[olivierlambert]

Yay!! \o/ Thanks for the reminder on this issue, it wasn't hard to fix but was kind of lost in the backlog ^^

[stormi]

@olivierlambert No it wasn't

[stormi]

To elaborate: it was already in progress, but delayed due to 1. the fact that I underestimated the impact (I thought we'd overwrite the file only when we'd modify it, not every update the package) and 2. lots of other priorities keeping the team busy.

But it's clearly the reminder, and the help in better diagnosing it, that allowed to have it re-prioritized and (partly, we've mostly bought time to fix it properly, by simply avoiding to touch the file at all at the moment) fixed so fast.