XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. john.c
    3. Posts
    J
    Offline
    • Profile
    • Following 0
    • Followers 0
    • Topics 16
    • Posts 367
    • Groups 0

    Posts

    Recent Best Controversial
    • RE: VMWARE to XCP-ng migration of 2TB disk

      @CyaVMware said:

      Have a simple question for someone.

      Looking at migrating a client to XCP. Currently they have a data disk that is exactly 2TB in VMware.

      My question is the limitation anything OVER 2TB, or is it 2TB AND above?
      I'd prefer not to use the VMware converter tool to shrink the disk to 1.99TB if I don't have to. 🙂

      VHD format has a strict limit of roughly 2TB (around 2040 GB), so if your client’s VM disks are larger than this then you’re going to have trouble with VHD format. However QCOW2 support is almost here (production ready) about to reach RC2 state, once this reaches stable you’ll be able to safely reach a max of around 16TB. That is if your using ext4 based SR, other Filesystems have different limits. If using a networked file sharing protocol service then, the limit of the host server’s filesystem applies. Though QCOW2 VDI has a limit of 16 TiB.

      posted in Migrate to XCP-ng
      J
      john.c
    • RE: 🛰️ XO 6: dedicated thread for all your feedback!

      @DustinB said:

      @john.c This is already viewable under each hosts in the pool.

      3eb9d466-59e6-423a-8733-b9655d734103-image.jpeg

      Then in that case the pools based view restructure can go ahead. Was just concerned about ensuring that it’s still possible to access individual hosts.

      posted in Xen Orchestra
      J
      john.c
    • RE: 🛰️ XO 6: dedicated thread for all your feedback!

      @DustinB said:

      @john.c said:

      @DustinB said:

      @olivierlambert https://feedback.vates.tech/posts/49/pool-view-rather-than-host-view

      So you’re interested in having it a bit more like when using VMware vSphere while, regarding VMs, hosts and pools? That could work, but still need a hosts view for each member of the pool. In the host’s view it needs to be able to see, what VMs are running on it.

      Also needs to be compatible with https://github.com/vatesfr/xen-orchestra/issues/9430. If not already synced with NetBox, the pool and hosts should be linked by UUID when synced. So in other words the structure needs to be pool is linked by UUID to host by its UUID and then the same for VMs. So the VMs are linked by UIID to the host UUID.

      Nothing in my post would restrict the view to only seeing VMs within a specific Pool, the view should be flexible enough to shows VMs in a pool.

      As it is now, you see all VMs that are running under a given host and then pool.

      This in and of itself could become tedious to manage/dig through.

      The potential remedy here would be to show all VMs in a pool, and on the given VM show the host where it's running.

      eed2974b-314b-4e25-a36a-15d88f8899a9-image.jpeg

      Even in VMware vSphere there’s a hosts view, that shows things specific to a particular host. This includes VMs running on it, or are currently halted or suspended on it. As well as patches missing from the host, etc. Though it more roundabout to get to it under VMware vSphere.

      matiasst created this issue in vatesfr/xen-orchestra

      open Link VMs to Host Devices via UUID in NetBox Plugin #9430

      posted in Xen Orchestra
      J
      john.c
    • RE: 🛰️ XO 6: dedicated thread for all your feedback!

      @DustinB said:

      @olivierlambert https://feedback.vates.tech/posts/49/pool-view-rather-than-host-view

      So you’re interested in having it a bit more like when using VMware vSphere while, regarding VMs, hosts and pools? That could work, but still need a hosts view for each member of the pool. In the host’s view it needs to be able to see, what VMs are running on it.

      Also needs to be compatible with https://github.com/vatesfr/xen-orchestra/issues/9430. If not already synced with NetBox, the pool and hosts should be linked by UUID when synced. So in other words the structure needs to be pool is linked by UUID to host by its UUID and then the same for VMs. So the VMs are linked by UIID to the host UUID.

      matiasst created this issue in vatesfr/xen-orchestra

      open Link VMs to Host Devices via UUID in NetBox Plugin #9430

      posted in Xen Orchestra
      J
      john.c
    • RE: IPMI Info Outlet Air temp missing.

      @acebmxer I’m sorry to say that if those Dells are at your workplace, the wrong edition of iDRAC was purchased. You see with at the very least iDRAC 9 access to the full granular IPMI sensor data, was placed behind an edition paywall by Dell Technologies. Outlet temperature is just one of the feeds, missing from the Enterprise or lower edition of iDRAC 9. You’ll get the temperature readings from the Dell iDRAC web browser based interface, but not IPMI with iDRAC 9 Enterprise.

      To obtain the full IPMI sensor data you need the Datacenter edition.

      posted in Xen Orchestra
      J
      john.c
    • RE: IPMI Info Outlet Air temp missing.

      @olivierlambert Worth having a word with Dell following checking, if IPMI can read under Dell the iDRAC edition, as this will affect readings reported by ipmi.

      posted in Xen Orchestra
      J
      john.c
    • RE: IPMI Info Outlet Air temp missing.

      @acebmxer said:

      The server i pulled info from was Dell r660 idrac 9 Enterprise.... My other servers are Dell r640s all have idrac 9 enterprise.

      My other hosts are older

      Screenshot 2026-03-03 155403.png

      That’s why the outlet temperature didn’t show, the edition isn’t high enough - see my linked PDF on the above, post. Without the Datacenter edition of iDRAC you’ll lose access to the full IPMI SDR sensor records, so won’t be available to the ipmi tool, thus the ipmi xapi plugin!

      posted in Xen Orchestra
      J
      john.c
    • RE: IPMI Info Outlet Air temp missing.

      @olivierlambert @acebmxer There’s readings for temperatures for outlets, connected to PSUs, especially in enterprise servers and/or workstations. It may not cover other outlets, but giving the ones going out through PSU may give some idea.

      Having Datacenter edition helps, if Dell iDRAC. Even better is using Redfish especially with same edition!

      The reason being as editions below Datacenter on Dell iDRAC have masking on subsets of the SDR sensor records, the highest edition unlocks the full SDR dataset.

      https://www.delltechnologies.com/asset/en-us/products/servers/industry-market/openmanage-portfolio-software-licensing-guide.pdf

      https://www.delltechnologies.com/asset/en-us/solutions/infrastructure-solutions/briefs-summaries/integrated_dellemc_remote_access_controller.pdf

      posted in Xen Orchestra
      J
      john.c
    • RE: Unable to connect to V5

      @olivierlambert said:

      I think it was fixed since, are you sure you are using an uptodate XO?

      Commit 0be23

      Anyway was directing attention to the GitHub issue, which was opened. The issue has another reference to it.

      posted in Xen Orchestra
      J
      john.c
    • Unable to connect to V5

      @pdonias @olivierlambert Ronivay has opened a bug report, GitHub issue about a problem with Xen Orchestra V5 vs V6 interface, when using an bound address (hostname) for the interface.

      https://github.com/vatesfr/xen-orchestra/issues/9500

      ronivay created this issue in vatesfr/xen-orchestra

      closed v5 interface doesn't work if hostname set #9500

      posted in Xen Orchestra
      J
      john.c
    • RE: Automation of all CURD operations

      @rama said:

      @olivierlambert thank you. but is it possible to keep tracking all the CURD operation like we have in terraform. but currently MCP have only Read tasks.

      Like if some new interns in my lab don't know about this and in this agentic framework if he/she need a VM's, delete or update it can be done very quick. it will save many hours.

      I hope this will be available in future or if you wish to do tell me how far it is.

      The plugin MCP Server is read only by design to keep using it safe, to have an MCP for reading and another for writing is best practice.

      If you desire to have a separate MCP server for the writing actions, feel free to suggest that in the feedback portal. You can even develop your own MCP server, which makes calls to the write side of the XO REST API.

      https://modelcontextprotocol.io/

      posted in REST API
      J
      john.c
    • RE: backup mail report says INTERRUPTED but it's not ?

      @flakpyro said in backup mail report says INTERRUPTED but it's not ?:

      @olivierlambert Using the prebuilt XOA appliance which reports:

      [08:39 23] xoa@xoa:~$ node  --version
      v20.18.3
      

      @majorp93 @pilow Can you please capture some heap snapshots from during backup runs of XOA via NodeJS?

      Then compare them to each other, they need to be in the following order:-

      1. Snapshot before backup
      2. Snapshot following first backup
      3. Snapshot following second backup
      4. Snapshot following third backup
      5. Snapshot following subsequent backups to get to Node.js OOM (or as close as you’re willing to risk)

      These will require that XOA (or XOCE) is started with Node.js heap snapshots enabled. Then open in a Chromium based browser the following url:-
      chrome://inspect

      The above URL will require using the browser’s DevTools features!

      Another option is to integrate and enable use of Clinic.js (clinic heapprofiler), or configure node to use node-heapdump when it reaches a threshold amount.

      Once your got those heap dumps your looking for the following:-

      • Object types that grow massively between the snapshots.
      • Large arrays or maps of backup-related objects (VMs, snapshots, jobs, tasks, etc.).
      • Retained objects whose “retainers” point to long-lived structures (global, caches, singletons).

      These will likely help to pin down what and where in the backup code, the memory leak is located.

      Once have these a heap snapshot diff showing which object type (or types) growing by a stated size per backup will finally help the Vates developers fix this issue.

      @florent I left the above for the original reporters of the memory leak issue, and/or yourselves.

      posted in Backup
      J
      john.c
    • RE: Migration from VMware to XCP-NG complete.

      @acebmxer Yes, definitely congratulations on migrating to Vates. You’ll be seeing improvements and innovation as a result. I’ve requested something which is about to come to fruition in Xen Orchestra, something which will possibly be of interest and use to you.

      https://github.com/vatesfr/xen-orchestra/pull/9519

      mpiton opened this pull request in vatesfr/xen-orchestra

      closed feat(mcp): add @xen-orchestra/mcp package #9519

      posted in Share your setup!
      J
      john.c
    • RE: Low end devices , share your experiences

      @Emmanuel-V said in Low end devices , share your experiences:

      I wonder to which extend we could lower dom0 memory, eventually reduce the set of features for this kind of usecases.
      I can definitely see a pool of machines (x86 or ARM) for industrial purpose where that could make sense, and that could be managed and update through XOA.

      Actually it’s not just the above, this technology (MicroVM) will also be of interest to hyperscalers down the road. As such I think that the handling of this should be an install time choice, as changing an already existing host to a MicroVM capable one can be messy. Doing it this way will enable a lot of capabilities for use with Xen Orchestra, in the handling of such MicroVMs.

      A hyperscaler already has this technology (Amazon AWS) with their Firecracker software technology.

      posted in Share your setup!
      J
      john.c
    • RE: DevOps Megathread: what you need and how we can help!

      @vates-11940382 Hi All at Vates,

      I just wanted to say a big thank you for the MCP support that’s now landing in Xen Orchestra. This is definitely a genuinely forward-thinking move, and it’s going to have a huge impact on how IaC tooling interacts with XCP-ng going forward.

      MCP gives XO a clean, structured, read-only interface that modern AI-assisted tools (e.g Pulumi Neo, Copilot, Claude, Cursor, etc.) can understand natively. That’s a massive step toward making XCP-ng an AI-visible, AI-navigable platform - something no other virtualisation stack is doing yet.

      What’s even more exciting is the long-term implication: this kind of openness and clarity is exactly what hyperscalers have been struggling with internally. If Vates continues down this path, it’s not unrealistic that MCP-native infrastructure could start attracting interest from much larger players - whether as customers, collaborators, or contributors.

      The combination of:

      • IaC
      • MCP
      • AI-assisted operations
      • and XCP-ng’s open architecture

      Puts Vates in a very strong position for the future.

      Thanks again for pushing this forward. It’s a big deal, and it’s going to unlock a lot of possibilities for the community.

      posted in Infrastructure as Code
      J
      john.c
    • RE: DevOps Megathread: what you need and how we can help!

      @nathanael-h As a further follow up the MCP agents include at minimum:-

      • Pulumi
      • MariaDB

      Anyway I would really appreciate having an MCP Server which, can be used when writing IaC to provision and configure your customers’ systems. Can it be made available during this year please?

      posted in Infrastructure as Code
      J
      john.c
    • RE: New Rust Xen guest tools

      @Tristis-Oris said in New Rust Xen guest tools:

      Not sure is it an issue. Agent itself is working.

      Welcome to Ubuntu 24.04.3 LTS (GNU/Linux 6.14.0-37-generic x86_64)

      systemctl status xen-guest-agent
      â—Ź xen-guest-agent.service - Xen guest agent
           Loaded: loaded (/usr/lib/systemd/system/xen-guest-agent.service; enabled; preset: enabled)
           Active: active (running) since Mon 2026-02-09 14:28:41 MSK; 36min ago
         Main PID: 959 (xen-guest-agent)
            Tasks: 17 (limit: 19041)
           Memory: 4.8M (peak: 6.7M)
              CPU: 76ms
           CGroup: /system.slice/xen-guest-agent.service
                   └─959 /usr/sbin/xen-guest-agent
      
      Feb 09 14:28:41 oris systemd[1]: Started xen-guest-agent.service - Xen guest agent.
      Feb 09 14:28:41 oris xen-guest-agent[959]: cannot parse yet os version Custom("24.04")
      

      Welcome to Ubuntu 24.04.3 LTS (GNU/Linux 6.17.0-14-generic x86_64)

      systemctl status xen-guest-agent
      â—Ź xen-guest-agent.service - Xen guest agent
           Loaded: loaded (/usr/lib/systemd/system/xen-guest-agent.service; enabled; preset: enabled)
           Active: active (running) since Mon 2026-02-09 15:04:31 MSK; 1s ago
         Main PID: 9058 (xen-guest-agent)
            Tasks: 17 (limit: 76999)
           Memory: 2.5M (peak: 4.7M)
              CPU: 32ms
           CGroup: /system.slice/xen-guest-agent.service
                   └─9058 /usr/sbin/xen-guest-agent
      
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      Feb 09 15:04:31 k3s xen-guest-agent[9058]: Specified IFLA_INET6_CONF NLA attribute holds more(most likely new kernel) data which is unknown to netlink-packet-route crate, expecting 236, got 240
      

      Is this a graphical (desktop) based VM running the agent?

      @olivierlambert If his VM is a desktop graphical Ubuntu then, it’s Kernel will be regularly updated with each new standard lifecycle point be an upgraded HWE kernel. So will be frequently getting this message, new info in the kernel. HWE kernels are default in modern Ubuntu desktop environments!

      posted in Development
      J
      john.c
    • RE: backup mail report says INTERRUPTED but it's not ?

      @MajorP93 said in backup mail report says INTERRUPTED but it's not ?:

      @john.c Considering how widely Node JS is being used out there I highly doubt that memory management in itself is broken in Node 22 and 24.
      If that would be the case it would have been covered by IT bloggers and most users would switch to using something else.
      Classifying memory management as unstable for the whole LTS branches 22 and 24 is something a LLM would do.
      I think it is more likely a XO + Node issue.
      @pilow already said that they are using XOA which (AFAIK) is still using Node 20.
      Even on Node 20 there seems to be some memory leak ongoing according to them which is why it being a "XO + Node" issue rather than a Node 22/24 being borked in general becomes even more likely.

      //EDIT: even if using Node 20 would improve anything here, sticking with it might not be the best idea as Node 20 will become EOL in April 2026.

      @bastien-nollet @florent @olivierlambert It takes placing enough stress on those certain areas, to trigger RSS spikes in NodeJS 22 and 24. It’s happened and/or happening to other developers who use NodeJS.

      Just to clarify a few things from the earlier AI‑generated reply:
      In this case we are not dealing with a kernel OOM kill. The log I attached in my first post clearly shows a Node‑level heap out‑of‑memory error. So statements like “No crash logs = kernel OOM” don’t apply here.
      That said, it is still worth looking into Node 22/24 memory behavior, but not because those LTS branches are “broken.” If Node’s memory management were fundamentally unstable, the entire ecosystem would be in chaos. Instead, what seems more likely is:
      XO’s backup workload + Node 22/24 = hitting a known memory‑management edge case.
      This is supported by the fact that even XOA (which uses Node 20) is showing signs of a slow leak according to @pilow. That strongly suggests the issue is not “Node 22/24 bad,” but rather:
      “XO + Node” interaction that becomes more visible under newer V8 versions.
      To support that, here are direct links to other developers and projects experiencing similar issues with Node 22+ memory behavior:
      🔗 1. Cribl’s deep dive into Node 22 memory regressions
      They observed significantly higher RSS and memory anomalies when upgrading from Node 20 → 22, and ended up contributing fixes upstream.
      “Understanding Node.js 22 memory behavior and our upstream contribution”
      https://cribl.io/blog/understanding-node-js-22-memory-behavior-and-our-upstream-contribution/ (cribl.io in Bing)
      This is one of the clearest real‑world examples of a production workload exposing V8 memory issues that didn’t appear in Node 20.
      đź”— 2. Node.js upstream issue: RetainedMaps memory leak in Node 22
      This is a confirmed V8‑level leak that affected Node 22 until fixed upstream.
      GitHub Issue #57412 — “Memory leak due to increasing RetainedMaps size in V8 (Fixed upstream)”
      https://github.com/nodejs/node/issues/57412 (github.com in Bing)
      This shows that Node 22+ did have real memory regressions, even if they don’t affect all workloads.
      đź”— 3. Broader discussions about increased RSS in modern Node/V8
      There are multiple reports of higher RSS and “apparent leaks” in Node 22+ under heavy async I/O, streaming, or buffer‑intensive workloads — which is exactly what XO’s backup pipeline does.
      Examples include:
      Matteo Collina’s posts on V8 memory behavior and GC tuning
      Various debugging guides for Node 22 memory regressions
      Reports from teams running high‑throughput streaming workloads
      These aren’t XO‑specific, but they show the pattern is real.

      Why this matters for XO?

      XO’s backup pipeline is unusually heavy for a Node application:

      • large streaming buffers
      • compression
      • encryption
      • S3 multipart uploads
      • high concurrency
      • long‑lived async chains

      This is exactly the kind of workload that tends to surface V8 memory issues that don’t appear in typical web servers or CLIs.

      And since Node 20 goes EOL in April 2026, XO will eventually need to run reliably on Node 22/24 or an alternative runtime.
      So the more accurate framing is:
      This is not a kernel OOM.
      This is a Node heap OOM, confirmed by the logs.
      Node 22/24 are not globally unstable, but they do have documented memory regressions and behavior changes.
      XO’s backup workload is heavy enough to expose those issues.
      Even Node 20 shows a slow leak in XOA, which strongly suggests a XO + Node interaction, not a Node‑only problem.
      Investigating Node 22/24 memory behavior is still worthwhile because XO recommends using the latest LTS.
      Long‑term, XO may need fixes, profiling, or architectural adjustments to run reliably on future Node versions.

      tunamagur0 created this issue in nodejs/node

      closed Memory leak due to increasing RetainedMaps size in V8 (Fixed upstream) #57412

      posted in Backup
      J
      john.c
    • RE: backup mail report says INTERRUPTED but it's not ?

      @Pilow said in backup mail report says INTERRUPTED but it's not ?:

      @MajorP93 probably irrelevant, but since end of december I noticed a memory-leak behavior on my XOA.

      I finally put up a job to restart it everyday 4.15am, otherwise at about 48h it was saturating it's RAM (8Gb...)

      no more problem with a reboot everyday but, something is cooking.

      Are you using NodeJS 22 or 24 for your instance of XO?

      As both of these have the issue, only it’s much worse for NodeJS 24. Only the NodeJS 20 as an LTS is the currently released one which has stable and predictable memory usage.

      posted in Backup
      J
      john.c