@SylvainB said in WORM Backups with XCP-ng / Xen Orchestra - Seeking Solutions & Experience:

Hello everyone,

I'm exploring options for implementing WORM (Write Once, Read Many) capabilities for my backups within my XCP-ng environment, specifically using Xen Orchestra.

My current setup:

• XCP-ng Version: 8.3
• Xen Orchestra Version: 5.106.4 (Stable)
• Intended Backup Target: Synology NAS

My primary goal is to ensure that my backup data, once written, becomes immutable for a defined retention period, offering protection against accidental deletion or ransomware attacks.

My questions are:

1. Does Xen Orchestra offer any native WORM features or integrations that I might be overlooking for its backup jobs?
2. If not directly, has anyone successfully implemented WORM backups with a similar perimeter (XCP-ng, Xen Orchestra, and potentially a Synology NAS or other storage solution)? I'm very interested in learning about your setup, the specific technologies you used (e.g., storage features, specific configurations), and any lessons learned or best practices.

Any insights, architectural recommendations, or shared experiences would be highly valuable.

Thank you in advance for your help!

Best regards,

SylvainB

You can setup in the Synology its WriteOnce feature, then configure the appropriate configuration settings, including retention ones.

This will prepare the appropriate WORM environment you’re looking for. It will work well due it being the equivalent to Vates solution and/or the S3 based one.

https://kb.synology.com/en-in/search?tags[]=WriteOnce

@wtdrisco said in Hosts compatibility:

As I am starting to build an environment for testing to replace VMWare, I had a question related to hardware.

When setting up multiple hosts, do these need to match the same specs (like VMWare?) for HA (moving VMs from host to host)?

I have several DELL R series servers, and some do not have the exact same CPU model or one has less memory than the other.

When setting up (HOST POOLS??) if I needed to migrate VMs, will this support different host configurations?

If the the hosts don't match by close enough, especially if their capabilities (e.g. instruction sets) and specifications. Then in the case of capabilities then the non-matching ones will be suppressed by XCP-ng so that they all match. Also when migrating the specifications, of hosts really need to match so that when VMs are placed on the hosts. There's no issues when live migrating between the each of the pool member hosts.

As the VMs expect at least a certain number of cores dependent on the hosts, and the number specified per each VM. If this number isn't met then that VM can't migrate to a specific host, which don't meet or exceed it.

@ThierryC01 said in XCP-ng 8.3 betas and RCs feedback :

@bleader Update successful, no issue so far. There is just a message appearing when starting the update: "Delta RPMs disabled because /usr/bin/applydeltarpm not installed."

All VM working for now.

That's harmless. It's just notifying that the system doesn't have deltarpm package installed. This package reduces the size of updates based on what's already installed.

@still_at_work said in First SMAPIv3 driver is available in preview:

Hello @olivierlambert ,

I am joining this topic as I have a few questions about SMAPIv3:

• Will it allow provisioning of VDIs larger than 2TB?

• Will it enable thin provisioning on iSCSI SRs?

Currently, the blockers I encounter are related to my iSCSI storage. This is a major differentiating factor compared to other vendors, and resolving these blockers would significantly increase your market share.

Thanks !

@still_at_work The size limit of the VDI is due to the file format used for these, which is VHD (https://en.wikipedia.org/wiki/VHD_(file_format)). This format can't support more than 2TB, it's known about and are dealing with the issue. It will likely result in a change or addition of a new VDI format likely to be qcow2 unless necessary software for VHDX format is fully open sourced and software for Xen is created which enables create, read, write and use of this format.

It's not a limitation of iSCSI as it also emerges with both NFS and SMB based connections.

@jasonnix The Linux distribution operating system restricts certain commands, operations and/or path (location) to the root account, as well as the location owner.

The path your trying to write to is just one such location as the location is read only to non-root users. If you really wish to write to /usr/local/src then root account is required.

If you use the root account to install then the directory and the Xen Orchestra files will be owner by root. You would need to create a non-root user for the xen orchestra server process. As well as change ownership of the Xen Orchestra files and folders to that non-root account. Though at least the ones which Xen Orchestra should be able to modify as required, when needed.

As well as provide the necessary sudo permissions for the necessary commands to that account.

Using root account to run exposes any vulnerabilities present to the capacity of being exploited as root (aka Administrator) privileges.

Also how good is your internet connection and your connection to GitHub? Cause you could be experiencing a time out packet loss connection issue.

@Octopuss said in ️ XO 6: dedicated thread for all your feedback!:

@olivierlambert Well ok, it's just that last time I tried to ask about stuff I was politely told off because I used the installation script instead of paying (that's not the literal formulation but it's basically what the person responding meant).

I know nothing about any blogs, I just check the main xcp website (FFS I am not even allowed to post links, what a forum...) from time to time, so I have no idea. I just updated the admin interfce for the first time since moving over from ESXi, and was puzzled why it looked the same as the "castrated" lite version when previously (which means v5, apparently) it was full of features. I had no idea it wasn't a completed product.

TL;DR: I am just an idiot who installed this few months ago and has no idea about the details. shrug

You were told off if the script was 3rd party because officially, the scripts may have made changes to the Xen Orchestra source code prior to compilation. They have a set of steps, for building from source which are able to receive support in the forums for. Anyway getting paid support if an organisation is best as their pro support is top notch!

With the blogs which are being referred to are under the “News” links. Which detail releases and other news worthy information.

You can post links but it does take some work, also make sure you’ve read the rules, also going through the introduction training which a forum function bot runs. Doing this will eventually earn you more access and feature functionality. It’ll help you get used to the forums functions.

The XO 6 is a minimum viable product currently, but is growing from there as features from v5 transitions to v6. If there’s features missing from v5 and v6 that are needed please enter them on the Feedback portal, so they can be voted on. It’s link is in several XOA update and release news posts (especially 6.0 and 6.1).

Read both these posts along with others along the way, it will help you get caught up on all of the Xen Orchestra releases and updates.

@olivierlambert I created the issue on GithHub. https://github.com/vatesfr/xen-orchestra/issues/7884

MrGrymReaper created this issue in vatesfr/xen-orchestra

open xo-server fix: Update acme-client npm and introduce support for External Account Binding (EAB) #7884

@peb2 said in XOA letsencrpyt module not setting acmeDomain:

Hello,

I'm new to Xen and XOA (migrating from Vmware) and I was wondering if anyone could help me with a problem.

I'm trying to use the Letsencrypt module as specified in:

https://github.com/vatesfr/xen-orchestra/blob/master/%40xen-orchestra/mixins/docs/SslCertificate.md?ref=xen-orchestra.com

However, the XOA doesn't seem to be passing the variable for 'acmeDomain' so I get a certificate without a CN.

Has anyone else seen this problem or know what my mistake may be?

Here's my /etc/xo-server/config.toml:

#=====================================================================

# HTTP proxy configuration used by xo-server to fetch resources on the
# Internet.
#
# See: https://github.com/TooTallNate/node-proxy-agent#maps-proxy-protocols-to-httpagent-implementations
#httpProxy = 'http://jsmith:qwerty@proxy.lan:3128'

# Configuration of the embedded HTTP server.
[http]
# If set to true, all HTTP traffic will be redirected to the first
# HTTPs configuration.

redirectToHttps = true

# Basic HTTP.
[[http.listen]]
# Port on which the server is listening on.
port = 80


# Basic HTTPS.
[https]
#
# You can find the list of possible options there https://nodejs.org/docs/latest/api/tls.html#tls.createServer
#
# The only difference is the presence of the certificate and the
# key.

hostname = '0.0.0.0'

[[http.listen]]
port = 443

# Whether to autogenerate a self signed certificate if the `cert` and `file`
# entries could not be found.
#
# Default: true
#autoCert = false
autoCert = true

# File containing the certificate (PEM format).
#
# If a chain of certificates authorities is needed, you may bundle
# them directly in the certificate.
#
# Note: the order of certificates does matter, your certificate
# should come first followed by the certificate of the above
# certificate authority up to the root.
#cert = '/etc/ssl/cert.pem'
cert = '/etc/ssl/xoa.cert.pem'

# File containing the private key (PEM format).
#
# If the key is encrypted, the passphrase will be asked at
# server startup.
#key = '/etc/ssl/key.pem'
key = '/etc/ssl/xoa.key.pem'

# ACME (e.g. Let's Encrypt, ZeroSSL) CA directory
#
# Specifies the URL to the ACME CA's directory.
#
# A identifier `provider/directory` can be passed instead of a URL, see the
# list of supported directories here: https://www.npmjs.com/package/acme-client#directory-urls
#
# Note that the application cannot detect that this value has changed.
#
# In that case delete the certificate and the key files, and restart the
# application to generate new ones.
#
# Default is 'letsencrypt/production'
acmeCa = 'zerossl/production'

# Domain for which the certificate should be created.
#
# This entry is required.
acmeDomain = 'xoa.mydomain.com'

# Optional email address which will be used for the certificate creation.
#
# It will be notified of any issues.
acmeEmail = 'myemail@mydomain.com'

The acmeDomain needs to be the actual Fully Qualified Domain Name (FQDN) which XOA uses. Also some verification and/or validation will need to be added either as a text file to the XOA appliance (in a web accessible directory) or as a DNS text record for XOA FQDN.

Also update the acmeEmail so its the actual email to be used for receiving the notification of issues. These don't look valid to me this is critical for them to be real, active and valid as the ACME Client uses these to send the certificate requests and renewals to the certificate authority!!

@Statitica said in Install XCP-ng in old HP ProLiant DL160 G6 (gen 6):

@ilu said in Install XCP-ng in old HP ProLiant DL160 G6 (gen 6):

@nick.lloyd Thankyou...i'll try the last version, reading forums people says HP was problematic, thats why i was asking for help.

Runs fine on my HP hardware, and was pretty easy to install.

@john.c said in Install XCP-ng in old HP ProLiant DL160 G6 (gen 6):

@ilu Though note that XCP-ng 8.3.0 when it goes LTS, will be getting a fix for the 2TB VDI limit. This will allow you to handle VMs which have large, disks.

You have a source for this? IIRC, the fix the the 2TB limit will fix a few other storage limitations too so I'm keen to know more.

https://xcp-ng.org/blog/2025/03/14/the-future-of-xcp-ng-lts/

@andsmith said in 8.3 USB Passthrough - Win Server 2019 BSOD:

@dinhngtu Sorry, I failed to mention this is a Windows Server 2019 VM, it appears that Windbg is a Windows 10/11 application. I've googled installing on Server 2019, but haven't had any luck getting it installed.

Update the AppX Package Deployment software via Microsoft Store or its equivalent in Windows Server 2019. Then once winget is updated (or installed) run the following command:

winget install Microsoft.WinDbg

@olivierlambert If this works for andsmith then it may be worth documenting this, for future users of Windows Server 2019, 2022 and possibly if required the upcoming Windows Server 2025.

@tjkreidl said:

@john.c @johnnezero Well, the Github reporsitory is in place already! Access it at:
https://github.com/tobiaskreidl/Citrix-Tobias-Kreidl-Collection
Feel free to find more material to add, as will I, and thank you kindly for the suggestions and support.

Is the repository supposed to just have the readme, or are there supposed to be other files (e.g. MDs, pics or the PDFs? Or is it in the releases section - which is currently empty?

@tjkreidl said:

@john.c Wow, that was amazing -- not sure why my searches were unsuccessful, but many thanks! I think Github might be a good option for putting these on-line as a more reliable spot. And, yes, preserving images is always a challenge.
I do hope some of that information may be useful to you and thanks much again for all your efforts, John!

A bit of AI archeology combined with Internet Archive Wayback Machine helped. The AI helped to surface the exact URL of the blog posts and these were fed into Wayback Machine. Used the snapshot from 2022 anything newer and they weren’t there anymore. By the way it may be worth sending a donation Internet Archive’s way (if not done so already), as without this your blog posts would be truly gone for good!!

Oh forgot to add GitHub and GitLab follows digital wills, so you can pass this off to another in a will, or generally be made read only while being kept up, when the time comes. So this hard work will be preserved even in the face of the end.

Just please let me and the rest of us know where it’s at when done!

@tjkreidl said:

@john.c Not found with the Wayback Machine, alas. Still not finding it anywhere else, but will keep looking!
It's a crying shame Citrix didn't preserve the treasure trove of old community blogs.

I did a bit of digging with the aid of AI and I’ve managed to uncover the original three blog posts of NUMA and references to UMA. If you do some more digging you may be able to uncover the rest, so it can be rewritten and/or updated, then be hosted somewhere that won’t go down so easily, or be lost as easily. If you do an update or write etc, may I suggest switching the images used to WebP or AVIF format, will seriously help file size while maintaining their quality (or even giving room for higher quality). Consider switching to SVG for diagrams rather than raster (or as the default). I’d suggest checking out to consider using Mermaid for diagrams (https://mermaid.ai/open-source/?utm_medium=hero&utm_campaign=variant_a&utm_source=mermaid_js). Maybe if you do a rewrite use markdown and something like Hugo, to generate it from the files to host on GitHub or some other pages providing repository (e.g. GitLab Pages or Codeberg Pages).

https://web.archive.org/web/20220527221535/https://www.mycugc.org/blogs/tobias-kreidl/2019/03/07/tale-of-two-servers-bios-settings-affect-apps-gpu

https://web.archive.org/web/20220527213026/https://www.mycugc.org/blogs/tobias-kreidl/2019/04/30/a-tale-of-two-servers-part-2

https://web.archive.org/web/20220527215004/https://www.mycugc.org/blogs/tobias-kreidl/2019/04/30/a-tale-of-two-servers-part-3

https://community.citrix.com/forums/topic/235895-xenserver-vm-citrix-worker-sizing-question/

https://xcp-ng.org/forum/topic/9359/cpu-provisioning

https://community.citrix.com/forums/topic/237493-memory-and-cpus-assigning-to-vms-in-order-to-obtain-maximum-performance-according-to-numa-topology/

https://community.citrix.com/forums/topic/241553-bios-power-performance-settings/

https://community.citrix.com/forums/topic/243640-citrix-hypervisor-performance-tips/

@tjkreidl said:

@johnnezero It would be also interesting to take UMA/NUMA into account as VMs -- in particular, VMs with vGPUS -- can run much more efficiently if they do not cross memory bank boundaries that span more than one CPU instance. On some Linux systems -- not sure about the one hosting XCP-ng -- you can even disable NUMA. Just an additional thought here. I published a number of years ago a three-part series "A Tale of Two Servers" discussing a number of related optimizations but alas, the Citrix blogs were eliminated and I'm snot sure where copies of these articles exist anymore. But there are plenty of articles on this, in particular by Frank Denneman, and also ones like the following;
https://indico.cern.ch/event/304944/contributions/1672535/attachments/578723/796898/numa.pdf
https://docs.xenserver.com/en-us/xenserver/9/numa.html

If you remember the URL and date maybe try the wayback machine of Internet Archive. They’re known to archive sites and articles wherever they can, may hold a copy that’s accessible.

@johnnezero said:

@john.c Thanks much, looking into it.
"Open-Source for the Win!"

Your welcome!

@johnnezero said:

@Pilow Sounds like an awesome idea. Send any details you may have on how to make plugins (if you know how that is).
Adding to the ToDo list - Thanks!

@johnnezero said:

@Pilow Sounds like an awesome idea. Send any details you may have on how to make plugins (if you know how that is).
Adding to the ToDo list - Thanks!

Looks like I can help out again just tracked down this past thread on the forums.

https://xcp-ng.org/forum/topic/7202/how-do-i-create-a-new-plugin

How’s your JavaScript (Typescript), JSON etc? These are the languages needed to develop plugins for Xen Orchestra.

@johnnezero Also there’s shortcut directories including one called cron.hourly, you can place a symlink (or hard link) to the script there. Cron will then execute the script without you needing to alter the crontab file.

Just drop .sh for using the shortcut directory as it will not run there otherwise.

@johnnezero said:

@john.c Great ideas (especially if Vates decides to bake something similar into XO someday) but may be getting too far into the weeds for now...

That’s okay. Just putting it out there - no rush! Any way to maintain Linux good practice relocate the conf file to /usr/local/etc/ (or /etc) then keep script in /usr/local/bin.

@johnnezero said:

@john.c Great idea, and done! ("Keep 'em coming")

Thanks for the change. By the way I meant when doing Vendor config and customer workflow config. To implement that requires nested config file. In other words the set-performance.conf, then a file for instance in a set-performance.conf.d/custom.conf (or similar). The custom.conf in the conf.d directory overrides the same property as well as section within set-performance.conf.

@johnnezero said:

WHAT: Automatically assigns CPU weights and I/O priorities based on assigned VM tag (i.e. replicating what vcenter did via resource pools etc.).

It would be even better if you could split the configuration section off, so that it’s in its own conf file. Would make it easier to manage, also if this ends up being used, by Vates in the Vates VMS software. There can then be a vendor recommended configuration with the option of customer’s own workflow based, configuration.