XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    • Profile
    • Following 2
    • Followers 0
    • Topics 10
    • Posts 124
    • Groups 0
    C Offline
    1. Home
    2. cg

    cg

    @cg

    52
    Reputation
    56
    Profile views
    124
    Posts
    0
    Followers
    2
    Following
    Joined
    Last Online

    cg Unfollow Follow
    • RE: XCP-ng 8.0.0 Beta now available!

      @stormi said in XCP-ng 8.0.0 Beta now available!:

      You could test with Citrix Hypervisor and then report it at https://bugs.xenserver.org
      Maybe they'll want to investigate even if the hardware is not officially supported. If someone can reproduce the issue on supported hardware, that would make things easier.

      Though I doubt it was woth the effort, as the support over there is well known (especially for not supported configurations etc.) I did it - for the reason of trying to improve stuff (and giving back to OSS community):
      https://bugs.xenserver.org/projects/XSO/issues/XSO-955

      posted in News
      C
      cg
    • RE: Citrix Hypervisor 8.0 landed

      @stormi dd stands for disk dump and does exactly that: Copy a stream of data.
      Fio however can be configured for precise workloads and read/write mixes, parallel workloads etc.

      So the first thing will only give you streamline benchmarks, what almost nobody cares about.
      The second can simulate realworld (VM/database...) workloads, where (controller) Caches and non magnetic storage (Flash, Optane, MRAM...) makes the real difference.
      Also use big amount of data, since caches can impact small ones extremely. Don't get me wrong: We need them and they can make huge differences, but as long as your benchmarks fully fit into them, it gives your nonsense/fake results. Also (consumer) SSDs start throttling after some 10 to a very few 100 GB of data written. Their caches fill up and they 'overheat'.

      You can spend days on benchmarks and how to do what. 😉

      posted in News
      C
      cg
    • RE: First SMAPIv3 driver is available in preview

      @olivierlambert said in First SMAPIv3 driver is available in preview:

      Hi,

      8.3 release is eating a lot of resources, so that's the opposite: when it's out, this will leave more time to move forward on SMAPIv3 🙂

      Lots of work means lots of changes, means: I'm exciting about it. Also sounds more like a 9.0, if that much work is going into it. 😉

      posted in Development
      C
      cg
    • RE: VM console borks with "Alt Gr" key

      The problem is still persistent or exisiting again.

      Exactly: You can use ALT GR once and get the symbol you need (@ |...) but afterwarts it's just putting weird symbols into the console.
      Dirty fix: Change the tab to whatever (general, stats...) and and back to console.

      Still pretty annoying and kicked me out of one of my servers for some time (thx fail2ban...). 😕

      posted in Xen Orchestra
      C
      cg
    • Blog Entry for 5.36

      https://xen-orchestra.com/blog/xen-orchestra-5-36/

      The part about ZFS:

      As there are many possibilities during the creation of a ZFS storage repository, you still need to use the command line (eg. zpool create zdata /dev/sdb) to create your SR.

      Check this documentation for more information about ZFS SR creation.

      Once the SR is created, you will have the option to select it during the SR creation directly in Xen Orchestra.

      Since it's confusing and technically wrong, it should rather be something like:

      As there are many possibilities during the creation of a ZFS storage repository, you still need to use the command line (eg. zpool create zdata /dev/sdb) to create a ZFS pool before creating the corresponding SR.

      Check this documentation for more information about ZFS SR creation.

      Once the ZFS pool is created and mounted, you will have the option to select the mountpoint during the SR creation directly in Xen Orchestra.

      ...and this forum is missing a "quote" button, as formating text in codeboxes doesn't work (and tags don't survive a linebreak, you have to mark it every line again).

      Regards,
      Christof

      posted in Xen Orchestra
      C
      cg
    • RE: Upgrade 8.2.1 -> 8.3 failed (manually fixed)

      @bleader

      IIRC I just "tried again".
      It failed 2 times, then I looked up the logs from other console, removed the file (which shouldn't be of any importance for our instance) and retried without reboot.

      I copied the whole installer-log to the usb stick before finshing the install. 🙂
      (Could actually be a good hint or even a menu-option for those, where the install fails and won't leave it on the harddrive - e.g. evaluating hardware)

      [  128.517356] ata1.00: exception Emask 0x0 SAct 0x800000 SErr 0x0 action 0x0
      [  128.517357] ata1.00: irq_stat 0x40000008
      [  128.517359] ata1.00: failed command: READ FPDMA QUEUED
      [  128.517362] ata1.00: cmd 60/80:b8:10:6c:d4/00:00:02:00:00/40 tag 23 ncq dma 65536 in
               res 41/40:10:80:6c:d4/00:00:02:00:00/00 Emask 0x409 (media error) <F>
      [  128.517363] ata1.00: status: { DRDY ERR }
      [  128.517364] ata1.00: error: { UNC }
      [  128.518008] ata1.00: configured for UDMA/133
      [  128.518018] sd 0:0:0:0: [sda] tag#23 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
      [  128.518020] sd 0:0:0:0: [sda] tag#23 Sense Key : Medium Error [current] 
      [  128.518021] sd 0:0:0:0: [sda] tag#23 Add. Sense: Unrecovered read error - auto reallocate failed
      [  128.518024] sd 0:0:0:0: [sda] tag#23 CDB: Read(10) 28 00 02 d4 6c 10 00 00 80 00
      [  128.518025] print_req_error: I/O error, dev sda, sector 47475840
      [  128.518039] ata1: EH complete
      [  128.581286] ata1.00: exception Emask 0x0 SAct 0x2000000 SErr 0x0 action 0x0
      [  128.581287] ata1.00: irq_stat 0x40000008
      [  128.581288] ata1.00: failed command: READ FPDMA QUEUED
      [  128.581291] ata1.00: cmd 60/08:c8:80:6c:d4/00:00:02:00:00/40 tag 25 ncq dma 4096 in
               res 41/40:08:80:6c:d4/00:00:02:00:00/00 Emask 0x409 (media error) <F>
      [  128.581292] ata1.00: status: { DRDY ERR }
      [  128.581293] ata1.00: error: { UNC }
      [  128.582111] ata1.00: configured for UDMA/133
      [  128.582117] sd 0:0:0:0: [sda] tag#25 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
      [  128.582118] sd 0:0:0:0: [sda] tag#25 Sense Key : Medium Error [current] 
      [  128.582119] sd 0:0:0:0: [sda] tag#25 Add. Sense: Unrecovered read error - auto reallocate failed
      [  128.582121] sd 0:0:0:0: [sda] tag#25 CDB: Read(10) 28 00 02 d4 6c 80 00 00 08 00
      [  128.582122] print_req_error: I/O error, dev sda, sector 47475840
      [  128.582133] ata1: EH complete
      [  128.629307] ata1.00: exception Emask 0x0 SAct 0x200 SErr 0x0 action 0x0
      [  128.629309] ata1.00: irq_stat 0x40000008
      [  128.629310] ata1.00: failed command: READ FPDMA QUEUED
      [  128.629313] ata1.00: cmd 60/08:48:80:6c:d4/00:00:02:00:00/40 tag 9 ncq dma 4096 in
               res 41/40:08:80:6c:d4/00:00:02:00:00/00 Emask 0x409 (media error) <F>
      [  128.629314] ata1.00: status: { DRDY ERR }
      [  128.629315] ata1.00: error: { UNC }
      [  128.630068] ata1.00: configured for UDMA/133
      [  128.630074] sd 0:0:0:0: [sda] tag#9 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
      [  128.630076] sd 0:0:0:0: [sda] tag#9 Sense Key : Medium Error [current] 
      [  128.630077] sd 0:0:0:0: [sda] tag#9 Add. Sense: Unrecovered read error - auto reallocate failed
      [  128.630078] sd 0:0:0:0: [sda] tag#9 CDB: Read(10) 28 00 02 d4 6c 80 00 00 08 00
      [  128.630079] print_req_error: I/O error, dev sda, sector 47475840
      [  128.630092] ata1: EH complete
      

      Indeed it looks like the SSD should be replaced.

      8.3 is running stable on this (and all other hosts, I upgraded) so far.
      It's a system at a UAS, running various student projects for several years now, coming from XenServer originally. I voluntarily maintain it. Thx for the hint!

      posted in XCP-ng
      C
      cg
    • RE: Windows 2025 Standard 24H2.11 (iso release of sept 25) crash on reboot with "INACCESSIBLE BOOT DEVICE 0x7B" in XCP 8.2.1 and XCP 8.3

      @dinhngtu I can't say on XCP-ng side, but it's likely linked to:
      August patch (and following), as Microsoft changed something to the NVMe stack.

      e.g.
      https://learn.microsoft.com/en-us/answers/questions/5536733/potential-ssd-detection-bug-in-windows-11-24h2-fol

      Google gives a lot about it. It seems that it most likely doesn't kill NVMes but can cause trouble.
      We have a few PCs becoming more unstable (BSODs) or even very slow after that upgrae.

      posted in XCP-ng
      C
      cg
    • RE: Backup solutions for XCP-ng

      @olivierlambert
      Okay, so bascially what I (and other people, I know, who are in a similar environment) need:

      • Tape backup is mandatory, as we're talking about (deduplicated, compressed) dozens of TB
      • Deduplication and good compression: duplicated, uncompressed ends in PB
      • All of that needs to scale! E.g. Commvault scales on 16 threads here even with only 1 - 2 tasks. (Can't say if even more without upgrade of CPU)
        (We need chains, of course, to go B2D2T -> dumping the dedup-store on tape as disaster recovery)
      • Applicationawareness (which can be done via agent - agentless is not always king or very important):
        -- MS Exchange, recovering datastores, mailboxes and even single mail items
        -- SQL Servers: MS SQL, MySQL (MariaDB)...
        -- Windows Active Directory Items

      Over here I also need to have the option to do backups only inside the VM via agent, as I can't snapshot them.
      You might have gotton me wrong: AFAIK quiesced backups always made a VM snapshot and submitted the request also to the VSS writer and then the VM/VHDs have been "read". In this case it will always fail, as there's no space on storage for such. Also Citrix discontinued that in their VM tools! XCP-ng yet lacks a proper tool maintenance with corresponding releases, which would be the only chance to keep that.

      As an idea for XOA:

      • Make it run on physical hardware, as VMs are too small for such
      • You already cooperate with HPE - let them run bare metal on ProLiants!
      • ZFS(oL) Could be your way to go to implement dedup and compression

      ...feel free to add, comment... whatever.

      posted in News
      C
      cg
    • RE: XCP-ng 8.2 updates announcements and testing

      Citrix also has a long history of problematic VM guest-tools. Race conditions with broken drivers, unbootable VMs, broken agents... It's a pain in the a**.
      If you like your VMs: Always make a snapshot of every VM before updating the tools.

      Is known why they remove the VSS-functionality? I don't get why they remove features (again).

      posted in News
      C
      cg
    • RE: XCP-ng 8.0.0 Beta now available!

      XCP-ng already ate a bunch of time, and I'm sure it'll continue for testing etc
      I'm Sysadmin with very limited coding ability - I'm definately out for that.

      I guess it would need someone being good at C#.NET. Also XAPI would probably need to know about it (what may already be, as you're implementing it into XOA)...

      posted in News
      C
      cg
    • RE: 9.2.385 snapshot/backup corruption fix: which versions were affected, and does it apply to XenServer VM Tools?

      @trelane said:
      When I turned off "Manage Citrix PV drivers via Windows Update" (has-vendor-device=false) to switch from XenServer VM Tools, Windows treated the NICs as new adapters.

      That bug had already cost me so much time. It has a looong history and seems to be "nailed in" when the VM had old tools installed at some point.
      I didn't have that problem with newer VMs that never had old tools tho.

      posted in Compute
      C
      cg
    • Booting Homeassistant (HAOS) Image breaks VM

      Hello!

      I have a very weird problem/bug!

      A while ago I imported the Homeassistant VMDK via XOA and stuffed it into an (Generic Linux UEFI) VM.
      It booted and runs flawlessly. Everything works - except backup via Commvault!
      Commvault fails reporting the VM could not be found. Support found the following inside the logfiles:

      4116  1984  05/01 21:00:29 58533 VSDisc::ProcessContent_VM() - virtual machine is not found using Name. Searching using both Name [HomeAssistant] and GUID [c3e276ed-1922-08a5-7f95-8b2d630e84ff]
      4116  1984  05/01 21:00:29 58533 CXenSDKWrapper::GetVMInfoForVM() - Error:Getting VM information failed.Die Eingabezeichenfolge hat das falsche Format.
      4116  1984  05/01 21:00:29 58533 VSDisc::ProcessContent_VM() - virtual machine: [c3e276ed-1922-08a5-7f95-8b2d630e84ff] discovery failed
      

      I assume "Die Eingabezeichenfolge hat das falsche Format." would be "The (input) string has the wrong format" in english.

      What I can: Discover the pool and select the VM to be taken into a backup job. So it somehow can see it, but it can't when running the actual job.
      Here's where things go really weird:
      I've spend quite some time, trying to narrow it down to a root cause. I've tried different VM templatesm, changed names and removed characters etc.
      Then I've interchanged the VDI with another VMs. Surprise:
      When I've interchanged HAOS VDI with a Debian Linux VMs OS VDI, the VM that cannot be backed up follows HAOS.
      Means: Now The VM named "Homeassistant" is found and backed up, but the VM called Debian-Test is not.
      So it turns out: Whenever a VM has booted HAOS, it can't be found during backup-discovery anymore. If it booted a "standard" Debian, the same VM is found again - until I change back to HAOS VDI.
      The support was clueless and we decided that I should try it over here: It looks like HAOS has to change something at the VM "skeleton" / metadata, that it can't be found otherwise.

      But how do I debug that further? How to find the difference and what exactly happens when it boots/runs?

      Greetings

      • Christof
      posted in Compute
      C
      cg
    • RE: 9.2.385 snapshot/backup corruption fix: which versions were affected, and does it apply to XenServer VM Tools?

      @trelane Seems to be a pretty annoying task. It says Upgrade from 9.2.350 is blocked, Need to apply "9.2.351 compatibility update" first. 😕

      I just read the release notes which are buggy:
      Important note for users upgrading from 9.2.350: You must first install the included XenTools-fix-9.2.351.msp hotfix to upgrade to 9.2.351 before installing 9.2.385.

      The zip-file does not containe that msp!
      ...it's a separate file to download.

      posted in Compute
      C
      cg
    • RE: USB-Passthrough does not survive reboot of VM

      @DustinB doesn't it use the exact same mechanism?
      I have to find out.

      posted in XCP-ng
      C
      cg
    • USB-Passthrough does not survive reboot of VM

      I had that many times with XCP-ng 8.2 and it felt like the problem is gone with 8.3, but it seems it's not (completely) gone.

      I can enable passthrough, add it to VM and it runs fine.
      I shutdown the VM for whatever reasons, recently it was just a reboot for Windows updates and the USB device got lost.

      What could be the reason, how to debug that?

      6487a635-933d-49ec-8ace-8a2b40e0a2f3-image.jpeg

      I can enable it again, shutdown VM (hotplug sadly still doesn't work till today?) attach and it's fine again.

      Regards

      • Christof
      posted in XCP-ng
      C
      cg
    • RE: Question on CPU masking with qemu and xen

      As I can't find it anywhere:
      What exactly is happening on join/leave?
      Is the masking changing at runtime and affecting every new started VM, does the pool need to be rebooted after an older server has been removed, to "unlock" all features?

      I have a pool with HPE DL380 Gen 9 and 10 where Gen 9 will replaced with Gen 12.

      So I assume it makes sense to remove 9 first, then add 12.
      But how exactly is the masking process working?
      Should I reboot Gen 10 first (shutting down all VMs) or add 12, move VMs over... or just shutdown and start all currently running VMs?

      In the early days (~XenServer 6) it had to be done manually by getting CPU features, calculating the commons and give it as parameter to join command.
      What I did not see though, is: How exactly is it working on removal and what's best practice.

      (also: should be added to documentations)

      posted in Compute
      C
      cg
    • RE: Booting to Dracut (I trusted ChatGPT)

      @AtaxyaNetwork did you ever think about "upgrading" the system?
      The installer should be able to detect and backup your current one, installl new and migrate all relevant data.
      That should exclude whatever home-brew-fiddlydiddly you did to your current installation.
      The reason you're seeing 2 almost identical partitions is exactly that: The way how upgrades work with XenServer/XCP-ng. One is your current system, the other one may contain a backup of a previous install (which can also be restored by the installer - if existing).

      As it's been mentioned: Neither trust USB-storage nor any AI without knowing what they suggest you to do. But you may have learned your AI-lesson: It can help, but it can end with a big pile of sh*t if you don't know what it's suggesting you. It's more A than I.

      posted in XCP-ng
      C
      cg
    • RE: Citrix or XCP-ng drivers for Windows Server 2022

      Small warning: XenTools are still a clusterfuck.
      On (allmost) all machines without our main environment it keeps the ip-configuration within the Realtek emulated NIC and never transfers it to the Xen device.
      It's fucked for long now - but doesn't matter if all VMs are getting their IPs via DHCP.

      Currently struggling with: Windows Server 2016 + XenTools/Management 9.4.2.
      At least it didn't break the whole VM by destroying the disk device drivers this round.

      tl;dr: Snapshot your VMs and have a documentation at hand, when upgrading the tools/drivers.

      posted in XCP-ng
      C
      cg
    • RE: Update Templates

      @bikemuch I registered an XOA-account to keep a few machines with XOA maintained. It's free, not much to loose.

      Many of us are looking forward for a 9.0 release, including a modern 5.x or even 6.x kernel, including all the developments and improvements, that have been made.
      Also things like 16TB+ disks on block storage (incl. thin provisioning) etc.

      If your company is running a bigger environment: You can put priorities on features as a paying customer. 😉

      posted in Xen Orchestra
      C
      cg
    • RE: Update Templates

      @bikemuch While you're technically not wrong about the impression, especially Linux distros usually work with flawlessly with the former version template.
      I also saw that there's a user-test running with updated templates including Debian 13 etc.
      It should be GA in about 2 weeks.

      posted in Xen Orchestra
      C
      cg