XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. billcouper
    3. Best
    B
    Offline
    • Profile
    • Following 0
    • Followers 0
    • Topics 2
    • Posts 9
    • Groups 0

    Posts

    Recent Best Controversial
    • RE: Please review - XCP-ng Reference Architecture

      If you run backups outside of business hours, any impact on pool hosts cpu/memory performance is likely irrelevant (and limited by how many resources the XO is provisioned with anyway). The bigger potential impact is likely on your production storage, which again could be irrelevant outside of business hours.

      However, if you want to perform backups more frequently and/or during business hours, in my experience the storage performance is the more likely to suffer noticeable impact. Unless your hosts are very highly utilized the additional cpu/memory load on a single VM shouldn't tip the scales. And at 500MB/sec your network shouldn't struggle either (I am assuming 10+Gbps links to get that speed).

      And regardless of backing up during or outside business hours, or how long your backup window is, always consider the restore times! Performance of backup storage is always low priority until something needs to be restored 🙂 If your backup takes 8 hours your restore will take 8 hours. Or longer. Don't cut any corners on backup storage, it is very important!

      posted in Share your setup!
      B
      billcouper
    • RE: Please review - XCP-ng Reference Architecture

      @TS79 In your backup/replication notes you say that remote to remote backup replication doesn't put any load on the pools, except that it does? The XO VM is doing the heavy lifting and will use CPU and Network resources, provided by the pool, to accomplish the replication. There are no "agent" like service running ON the remote that can offload the transfer.

      If you haven't locked in your production storage, and have <25ms latency between your sites, consider PureStorage Flash Array //X. We use these for production storage (and backups on //E arrays)... Long story short, if you deploy a PureStorage Flash Array into both sites, you can set them up in ActiveCluster, like we have done. You can then split your production cluster between the two sites. Hosts in both sites have paths to storage on each side, but use the array closest to them - the arrays then replicate the writes between themselves. It's truly active-active synchronous replication. This allows a design where the failover between sites is automated. If one of the sites goes down, the workloads will automatically restart in the other site. There's a lot of network design that goes into this, but it's definitely possible as we're doing it.

      posted in Share your setup!
      B
      billcouper
    • RE: Health alerts/alarms

      @olivierlambert There already seem to be some alerts configured inside the xcp-ng servers? Once I configure email alerts using XCP-ng center (just setting the smtp server and email addresses) I do get notifications about items like multipath and bond status (eg path lost, nic disconnected, bond status changed, etc). I had thought that Initially it would be good if we can simply enable the built in alerts using xoa, the same way that xcp-ng center enables them.

      If you were going to develop a standalone alarm/alert system in XOA, that's a giant kettle of fish.

      These are just some of the things I'd like to see alerts for (and be able to trigger some type of action, eg 'send email' or an api call).

      • NIC down (that was previously up)
      • Bond link count changed
      • Host unresponsive
      • Host patches required
      • Host restart required
      • Host memory usage
      • Pool memory usage
      • SR path lost (one of multiple paths went down)
      • SR all paths down
      • SR not connected to all pool hosts
      • SR capacity (with configurable thresholds as % or GB)
      • SR latency
      • Remote storage connectivity lost
      • Remote storage capacity (% or GB)
      • VM CPU co-stop (or xen equivelant)
      • VM Management Agent detected
      • VM Snapshot count
      • VDI remains attached to control domain after backup

      Being able to apply the monitoring granularly and hierarchically are very important. For example, I might set a global latency alarm that applies to all SRs. But maybe one of my pools is 'special' so I apply a stricter latency alarm at that pool level (the rest of the pools can use the global threshold). Then inside that pool, one SR in particular might be unique and need it's own latency alarm configured at the SR level.
      So the inheritance of the alarms for SRs should be Global > Pool > SR

      So thinking of this type of granular requirement with hierarchical inheritance, XO6 seems like it could be possible, due to the new tree view which could potentially allow setting alert policies at each level.

      posted in Xen Orchestra
      B
      billcouper
    • RE: Health alerts/alarms

      I have done some more testing with this.

      I am able to generate test alerts/alarms from the xcp-ng cli:

      xe message-create pool-uuid=<my-pool-uuid> name="More Test" body="More Test Message. Very Message. Much Test." priority=3
      

      I found that if I use XCP-ng Center and configure a pool to send email notifications, that configuration is saved to each host in the pool - the email alerts are sent even when XCP-ng Center application is no longer running. I get the notifications that I am after.

      As I am instructing my engineers to move away from using XCP-ng Center and instead do everything via XOA, it would be great if this pool-wide alert email notifications configuration can be achieved using XOA.

      posted in Xen Orchestra
      B
      billcouper
    • Health alerts/alarms

      Hi all,

      I can only see performance and backup alerts in XOA. How do I get alerts/alarms for health related events?

      For example, I would like to know when "the bad things" happen in my environment, such as... Network configuration issue... Multipath Alert... Bond status... etc.

      I see in XCP-ng Center that it raises Alerts for these type of events, and can send emails too... however I believe I need to leave the software running for this to work? If I close the app, log off the machine, or it reboots for updates or something, then I no longer have email alerts for these important health events... is that correct?

      Regards,
      Bill

      posted in Xen Orchestra
      B
      billcouper