XO Pool Management
-
I'm in the early stages of testing XCP/XO (on Sources) for possible VMW migration in the future. I have to say...so far, I'm not at all impressed. I haven't even started creating VMs yet and am having all sorts of "bugginess"..just with Pools.
First, I like the fact when adding Hosts to a Pool, and making Network or Storage changes/adds, the change is applied to all Hosts in a Pool. Cool! Aside from that, Pool mgmt just seems to be buggy. Any explanation is appreciated.
My test setup is simplistic:
2 Dell Proliant DL360 G9 Hosts, XCP-NG 8.3 installed via ISO. Easy enough. 2 HBA cards using just 1 port on each for now...1 10Gb port for Mgmt & other networks, and 1 10Gb port on 2nd card for iSCSI storage connection
An Ubuntu 24.04 VM with XO via Sources installed. Again, fairly easy. All good so far.
I then was looking at Storage steps (adding iSCSI) and configuring my Networking to see my SAN & adding an IP to a HBA was not really intuitive at all. Found a post on here to help me through that...after 2hrs of clicking around
So, I think I have my Storage and Networking squared away. Now just needed to create a Pool and get my Hosts in one. This has been klunky to say the least; 1. in how it's done, then 2. Host behavior when in a Pool.
a. Add 2 Hosts to XO. Ok
b. Click Host1 and "go into" its Pool. Change the "Name" to give a more appropriate Pool name for multiple Hosts. Ok (but still a bit klunky in how this works)
c. When in this new Pool, add my Host2 to it. Ok (so far)
d. My Master (Host1) seems to be fine in the Pool; my newly added Slave (Host2) is "yellow" status. It says 'busy'. I go to Advanced tab to try to Enable and it won't (why it isn't already Enabled, I don't know). Host2 also appears to be in Maint Mode. Why? If I click on "Disable Maint Mode" button, it has the same behavior as when I try the "Enable" button...nothing. I can't even 'Detach' this Host2 from the Pool And, this is the 2nd time I've tried these steps. When I was clicking around the 1st time with Pools, my Host2 showed all RAM used (nothing is on any Host) and showed as "halted" or "offline". I ended up reinstalling XCP 8.3 and re-adding to XO. Took care of those error things, but again...still not acting right this time either. Also, does Slave Hosts take the root pwd of the Master? I'm noticing that behavior as well. How is that possible? That isn't a good thing imo. And now, it appears my Host2 lost it's mgmt nic configuration...so another reinstall will seemingly need to happen 
And again, I'm just in the setup pieces...not even testing VM creation and management. I really do want this to work ok so I can eventually leave VMW, but I'm not at all confident in this h/v solution to this point. Isn't this supposed to be "stable"? I have my doubts
Any suggestions appreciated... -
I have been using XO & XCP-ng for years now for many of our commercial clients and it's a very solid platform.
While there is a lot of good documentation for XO & XCP-ng first starting out with a new virtualization platform can be a steep learning curve which is why I have a series of videos that are designed to help people get started on the platform. Hope you find them helpful.Your Complete Open Source Virtualization Guide For Getting Started With XCP-ng & Xen Orchestra
https://youtu.be/2wMmSm_ZeZ4?si=6RsiFiENct8fxurwLevel Up Your XCP-ng & XO Knowledge: Tips for System Management & Recovering Hosts From Failure
https://youtu.be/WvXTq3adN08?si=ruaG2tMuC9iBSmTq -
@lawrencesystems Hi Tom....I actually have watched several of your vids. Thanks for them! I've been in virtualization for quite some time, so I have a decent general understanding; altho, I understand each h/v has its own quirks/bumps. I have also been reading thru both XCP and XO docs. There isn't much details in them tbh. But again, there isn't too much in Xen central mgmt tools either.
I know Xen's been around a while, as well as its central mgmt (XenCenter), but XO hasn't (really). Pools in XO seem to be kind of klunky; at least it's been that way to me playing around in it. I've rebuilt my 2 Hosts again for a 3rd time and re-added them to XO. I have yet to consolidate them into 1 single Pool again though. I'll re-attempt things again on Mon. One thing which "may" have caused issues I wasn't fully aware of -> patch installs. I did install them initially, but the msg that pops up says the Host reboots, and in actuality..it doesn't. This time around, I did manually reboot them. Maybe that caused issues?...not sure. Although...I did end up rebooting one of my Hosts multiple times in my prev testings, but that didn't rectify any issues I was having (but I didn't reboot my Master).
I'll update here if I continue to have issues. Thanks!
-
Yes, always have all systems to full patch level before adding them to pools.
Also, I don't recommend using Xencenter just use XO.
For clarification the platform has been around and in use for a while. The Xen hypervisor was first released in 2003 then Citrix Xensever around 2007 with XO first being released in 2013. XO has been around since 2013 first to manage Citrix and then Vates released XCP-ng in 2017. XO Still works with both platforms.
-
@lawrencesystems Hi Tom...yessir...I'm just testing out XO. I do have XC and just "looked" at it, but am not doing any configurations, etc in it. It was more to see what it looked like. Again, all my testing will be within XO. Thanks.
I knew (from your vids, actually) Xen has been around awhile, but I guess I wasn't aware XO has been. Thought that's only been out about 5yrs or so. I'm still frustrated at my testings so far, but not giving up. Hopefully my continued testing come next wk will result in more positive outcomes fingers crossed
Thanks for reaching out!
-
First release of Xen Orchestra (based on
xend, so oooold Xen API) was during summer 2009. Rewriting for XAPI is done back in 2013. It's been a bit longer than few years
Also, I can guarantee that if everybody had a similar experience than yours, nobody would have adopted our stack. I know it doesn't help you directly, but there's so many specific and external factors that can make your experience painful in your environment. There's no reason to reinstall XCP-ng, and what you think your network issue is, it's likely the pool member simply cannot joining the master.
Anyway, I hope the community will assist you as Tom started, so you can get on your feet. Thanks for keep trying, your feedback is important for us.
-
@olivierlambert - understood..which is why I'm sticking with it to this point.
I wonder if some of my network issue was due to nic adapter Name & Description changing? After placing the Hosts in the Pool, I was renaming the NICs to something more relevant, like "mgmt" and "iscsi", as well as adding a Description. I would think that shouldn't be an issue, but was the only "extra" thing I did to the Hosts after adding them to the Pool. shrug
Thanks for the info/feedback.
-
@olivierlambert - if I do run into such an issue again with my Hosts in the Pool and it's "offline" or "halted" or whatever, I can't "detach" it from the Pool, I can't disable Maint Mode (I never put it in this Mode, nor did I disable it...even tho it showed in these states)... how can I get the Host back to normal without a complete XCP reinstall? For the Host that lost network, I tried to reconfigure/reset it from xsconsole, but after I went through that process..added the NIC info (IP, etc) and the Host rebooted, when it got back to the Console, there NIC settings again didn't show/were lost.
I think I saw on another post where you told someone to delete the Local Storage, then re-add? But in XO, all of the Storage showed as disconnected? With no NIC or settings, I guess the only way to do so was from iLO console?
-
Sounds like you have some issues either because you are changing the IP's after you setup the pool or you are having
I cover how to deal with host failures in this video:
Level Up Your XCP-ng & XO Knowledge: Tips for System Management & Recovering Hosts From Failure
https://youtu.be/WvXTq3adN08?si=ruaG2tMuC9iBSmTq -
@lawrencesystems I'll give that vid a watch & see if it has any info for what I'm experiencing. Thanks!
-
@lawrencesystems Well...I liked your video...showed me features XO has similar to vCenter (UUID mgmt, Tagging, Maint Mode behavior, etc).
Nothing there really pertained to the netwk issue I experienced. My Slave host somehow completely lost its network & I couldn't recover it (tried to from the Console). Because XO could no longer 'see' this Host, I couldn't even Detach it from the Pool. Maybe if there was some way to 'reset' the Host (manual Pool removal), I then wouldn't have needed to completely reinstall XCP. But, this process isn't in the documentation, and I didn't really search for it online...thought it would just be quicker to reinstall and start from scratch again.
And again...I didn't do much to either Host. All I was doing was changing Network Names and adding Descriptions. Issue may have been I was doing directly on a Host and not at the Pool level, although I forgot where specifically I was attempting this. In your video you just shared, you mention making any & all Network changes on Hosts directly before adding to the Pool...is that correct? Don't make NIC changes within a Pool?
Thanks. -
@coolsport00 From your master over SSH (or console)
xe host-list
xe forget-host uuid=<Slave-UUID> -
@DustinB Thanks Dustin. If my Slave loses network again...I'll run that cmd. Hopefully not though!
Question if I may - does it matter where I configure my Host networking? Meaning (not my Mgmt ntwk..that's upon install), but if I want to change Network names/desc's, do I do that after adding Hosts to a Pool, or just go ahead and do it on the Hosts before adding them to a Pool? When I had my issue mentioned earlier above..I literally hadn't done anything but added 1 Host to my other in a Pool and just began homogenizing Networks, as well as adding a couple other VLAN networks. That was it..then bam!...issues on my Slave.
Thanks. -
@coolsport00 said in XO Pool Management:
@DustinB Thanks Dustin. If my Slave loses network again...I'll run that cmd. Hopefully not though!
Question if I may - does it matter where I configure my Host networking? Meaning (not my Mgmt ntwk..that's upon install), but if I want to change Network names/desc's, do I do that after adding Hosts to a Pool, or just go ahead and do it on the Hosts before adding them to a Pool? When I had my issue mentioned earlier above..I literally hadn't done anything but added 1 Host to my other in a Pool and just began homogenizing Networks, as well as adding a couple other VLAN networks. That was it..then bam!...issues on my Slave.
Thanks.I've always configured and setup my hosts into a pool, then went in and made any descriptive changes once the environment is setup.
Making changes on a per host basis doesn't really work or make any sense as they would be superseded by changes made at the pool.
-
@DustinB Yep...makes total sense, especially when you have multiple Hosts. Make 1 config or change and let Pools propagate to all Pool Hosts is my thought too. I'm just trying to pinpoint how my Slave Host got corrupted...if there was some issue configuring a Host directly vs it being in a Pool.
Thanks. -
@coolsport00 Ok...just found in XCP docs, it says you "should" rename NICs before joining Hosts to a Pool:
https://docs.xcp-ng.org/networking/#renaming-nicsThough, that's for XCP directly and says the eth's need to be down to make the change. I'm not entirely sure this is the same process when doing via XO? Is that really needed?
I would think a simple Name/Descr change doesn't require a downed eth and changes can simply be done in a XO Pool? -
@coolsport00 These are two different features. The XCP one involves renaming the physical NICs. Any changes you make in XO doesn't rename the physical NIC.
-
@Danp Ah ok, understood. Does XCP use netplan for Linux networking mgmt? I know how to rename pNics in that. Maybe I just do that and it'd be easier? I rename pNics for other Linux devices I use for Veeam backup components; so it would be no big deal to do here if possible before I even start adding the XCP Hosts to XO. But, I guess as long as "logically" within XO I see the pNic name what I want so I know what it is...all good.
Thanks. -
@coolsport00 Please stick with the documented method to be safe.
-
@Danp Understood...and that's what I did (in XO). Thanks!
