XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    1. Home
    2. johnnezero
    johnnezeroJ Offline
    • Profile
    • Following 8
    • Followers 0
    • Topics 3
    • Posts 23
    • Groups 0

    johnnezero

    @johnnezero

    10
    Reputation
    17
    Profile views
    23
    Posts
    0
    Followers
    8
    Following
    Joined
    Last Online

    johnnezero Unfollow Follow

    Best posts made by johnnezero

    • Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

      WHAT: This post is dedicated to preserving the valuable server administration guide content produced by @tjkreidl, originally titled "A Tale of Two Servers." These articles were first published in 2019 on the Citrix Blogs, which are unfortunately no longer available.

      KUDOS: Special thanks to @john.c for performing some incredible web sleuthing to recover this content, and to @tjkreidl for creating these essential guides in the first place — as well as for "Breathing Life" back into them by highlighting their importance before they disappeared forever into the ferocious "Bit-Bucket."

      Series Overview & Quick Reference

      If you're looking for a high-level summary of the concepts covered in this series — including specific XCP-ng/XO commands for BIOS power management, GPU scheduling, and NUMA inspection — start here:
      Quick Reference & Summary Guide

      The Complete Series

      =========================================

      Part 1: How BIOS Settings Can Affect Your Apps and GPU Performance:

      A deep dive into CPU Power Management (OS DBPM vs. System DBPM) and Turbo mode.
      Read Article

      Part 2: GPU Settings and Advanced BIOS Tuning

      Exploring Uncore frequency, C1E states, and NVIDIA GPU Scheduler modes (Best Effort vs. Equal Share).
      Read Article

      Part 3: NUMA, CPUs, Sockets/Cores, and VM Performance

      Understanding vNUMA, vCPU oversubscription, and the importance of VM startup order for optimal memory placement.
      Read Article

      Archived for the XCP-ng community. Questions or additional insights? Please discuss below!

      posted in Hardware
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      @poddingue FYI: I posted the "Feature Request" to the feedback channel here as you suggested.
      Hopefully if enough people were used to the way VMware allowed "multiple granular permissions to be set per VM", it will get some traction. Thanks again for pointing me in the right direction.
      Happy Day 🙂

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      @poddingue Thank you for the feedback, and suggestion. I will post a request regarding the one-ACL-per-VM issue, as I know it's an issue with our current 600+ VM VMware-to-XCP-ng migration project. Happy Day 🙂

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      @tjkreidl Found it. I re-posted the topic back to the hardware category (and linked the pdf's to the #tobiaskreidl github) here:
      Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      @john.c Will do - and kudos to you on some phenomenal sleuthing work (and to @tjkreidl for sparking ithis contents recall from an early "bit-bucket" grave)! Happy Day All!

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      @john.c FYI: Bundling/done (Plugin/To come...)

      posted in Management
      johnnezeroJ
      johnnezero
    • Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      UPDATE: Tag-Based Automation Plugin is now available

      ####(click here for new thread)

      ===================================

      IMPORTANT: THE FOLLOWING IS OLD/LEGACY CODE

      (Use the link above link for the new XO plugin instead)

      UPDATE: Tag-Based-Automation-Bundle v1.0

      WHAT'S NEW: This update introduces the "Tag-Based-Automation" bundle -- a modular, install.sh-based package that replaces the original standalone set-performance.sh script.

      This bundle is the foundation for the upcoming "Tag-Based-Automation" plugin for Xen Orchestra (if/when all that gets worked out with Vates).
      For now it is a production-ready shell-based bundle that installs cleanly on all XCP-ng pool hosts, and will only run on the current Pool Master.

      The goal is simple: Metadata (Tags) should drive Infrastructure State.

      STANDARD DISCLAIMER HERE

      This software is provided "AS-IS" without any express or implied warranty. While these scripts are being used in a production environment managing VMs, you should always review the code and test it in a non-production environment before full deployment.
      Note: The scripts are designed to ONLY take action on VMs with specific tags assigned -- untagged VMs are never touched.
      That said, as always - your mileage may vary...

      WHAT IS IN THE BUNDLE?

      [ACTIVE]      set-performance.sh
                    Automated enforcement of CPU weights and I/O priorities
                    based on assigned VM tags. Runs via cron. Prevents
                    configuration drift across your entire pool.
      
      [COMING SOON] Manage Tags via CSV
                    Bulk VM metadata management using a CSV as Source of
                    Truth workflow. Export your VM list, edit tags in Excel,
                    sync back to the pool in one command.
      
      [COMING SOON] set-permissions.sh
                    Automated management of Xen Orchestra Resource Sets and
                    user permissions -- driven entirely by your existing VM
                    tags. No manual Resource Set updates ever again.
      

      CHANGE-LOG

      v1.0 - 2026-05-16 (Bundle Release)
        - Modular Architecture: Repackaged as a modular bundle for easier
          updates and future plugin development
        - One-Shot Installer: install.sh handles directory creation,
          permissions, and legacy standalone cleanup detection
        - Orchestrator: main.sh manages module execution and cron
          scheduling via symlink -- no manual crontab editing required
        - Dual Config System: default.conf (never edit) + custom.conf
          (your pool settings) -- your settings are preserved on re-install
        - NFS Integration: Optional log aggregation for centralized
          reporting across multiple pools
        - Master Host Check: Robust UUID comparison ensures script only
          runs on the Pool Master -- safe to deploy on all hosts
        - Foundation for upcoming Xen Orchestra plugin
      
      v3.2 - 2026-05-14 (Final Standalone Release)
        - Fixed Master Host Check: Replaced invalid xe host-is-master
          with correct UUID comparison method
      
      v3.1 - 2026-05-14
        - Config Separation: Moved to conf.d/ override pattern
        - Split into default.conf and custom.conf
      
      v3.0 - 2026-05-14
        - TAG_SUFFIX support for multi-pool deployments
        - Auto-scheduling via initialize -- no manual crontab needed
      
      v2.0 - 2026-05-14
        - External configuration file
        - Master Host Check (first attempt)
        - Main log + summary log
        - Per-tier summary counters
      
      v1.0 - Initial Release
        - Tag-based CPU weight and I/O priority enforcement
        - Four tiers: 0-core, 1-high, 2-normal, 3-low
        - Network QoS cap (100Mbps) for 3-low tagged VMs
      

      CLEANUP: REMOVING THE OLD STANDALONE VERSION (If installed)

      If you installed the original standalone script, run these first:
      
        rm -f /usr/local/bin/set-performance.sh
        rm -rf /usr/local/etc/set-performance.conf.d
        rm -f /etc/cron.hourly/set-performance
        rm -f /etc/cron.daily/set-performance
        rm -f /etc/cron.weekly/set-performance
        rm -f /etc/cron.monthly/set-performance
      

      DOWNLOAD (Recommended)

      Download the v1.0 bundle directly from GitHub:
      https://github.com/johnezero/xo-tag-automation/releases/download/v1.0/tag-automation-v1.0.tar.gz

      INSTALLING THE BUNDLE

      STEP 1: Download and extract on your Pool Master

      wget https://github.com/johnezero/xo-tag-automation/releases/download/v1.0/tag-automation-v1.0.tar.gz
      tar -xzvf tag-automation-v1.0.tar.gz
      cd tag-automation-bundle/

      -- OR --

      Download to your workstation and upload via SCP or WinSCP:
      scp tag-automation-v1.0.tar.gz root@your-pool-master:/root/
      Then on your Pool Master:
      tar -xzvf tag-automation-v1.0.tar.gz
      cd tag-automation-bundle/

      STEP 2: Run the installer
      chmod +x install.sh
      ./install.sh

      The installer will:
      - Detect and warn about any legacy standalone components
      - Create all required directories
      - Deploy all scripts and config files
      - Preserve any existing custom.conf on re-install
      - Verify your NFS code path if applicable

      STEP 3: Edit your pool-specific config:
      *** ACTION REQUIRED -- DO NOT SKIP ***
      vi /usr/local/etc/tag-automation/conf.d/custom.conf

      Set your TAG_SUFFIX to match your pool:
      POOL-1 --> TAG_SUFFIX="-1"
      POOL-2 --> TAG_SUFFIX="-2"
      Single pool --> TAG_SUFFIX=""

      STEP 4: Initialize the bundle (REQUIRED):
      *** ACTION REQUIRED -- DO NOT SKIP ***
      /usr/local/bin/tag-automation/main.sh initialize

      Expected output:
      [OK] Symlink created : /etc/cron.hourly/tag-automation
      [OK] Schedule : hourly
      [OK] Tag suffix : (none)
      [OK] Active tags : 0-core, 1-high, 2-normal, 3-low
      [OK] Performance : true
      [--] Permissions : Coming Soon

      *** Bundle will NOT run until initialize is completed ***

      STEP 5: Verify it is working
      /usr/local/bin/tag-automation/main.sh
      tail /var/log/tag-automation.log
      tail /var/log/tag-automation-summary.log

      THE FILES (i.e. The hard way)

      1-of-5: install.sh
      Note: The install bundle can e located anywhere (i.e. /root/tag-automation-bundle)

      --------------------------------------------
      
      #!/bin/bash
      
      # ============================================
      # Tag-Based-Automation -- install.sh (v1.1)
      # One-shot installer for the full bundle
      # ============================================
      
      BUNDLE_DIR="$(dirname "$0")"
      INSTALL_BIN="/usr/local/bin"
      INSTALL_CONF="/usr/local/etc"
      
      echo ""
      echo "Tag-Based-Automation -- Installer v1.1"
      echo "============================================"
      echo ""
      
      if [ "$(id -u)" != "0" ]; then
          echo "Error: This installer must be run as root."
          exit 1
      fi
      
      if ! command -v xe &>/dev/null; then
          echo "Error: xe command not found."
          echo "This installer must be run on an XCP-ng Pool Master."
          exit 1
      fi
      
      echo "Checking for legacy standalone components..."
      LEGACY_FOUND=false
      
      [ -f "/usr/local/bin/set-performance.sh" ] && \
          echo "  [!] Legacy script found: /usr/local/bin/set-performance.sh" && \
          LEGACY_FOUND=true
      
      [ -d "/usr/local/etc/set-performance.conf.d" ] && \
          echo "  [!] Legacy config dir found: /usr/local/etc/set-performance.conf.d" && \
          LEGACY_FOUND=true
      
      crontab -l 2>/dev/null | grep -q "set-performance.sh" && \
          echo "  [!] Legacy crontab entry detected." && \
          LEGACY_FOUND=true
      
      LEGACY_CRONS=$(find /etc/cron.hourly /etc/cron.daily /etc/cron.weekly \
          /etc/cron.monthly -name "set-performance" 2>/dev/null)
      if [ -n "$LEGACY_CRONS" ]; then
          echo "  [!] Legacy cron symlinks detected:"
          echo "$LEGACY_CRONS" | sed 's/^/      - /'
          LEGACY_FOUND=true
      fi
      
      if [ "$LEGACY_FOUND" = true ]; then
          echo ""
          echo "-----------------------------------------------------------------------"
          echo "CAUTION: Legacy standalone components detected."
          echo "Please remove them before proceeding:"
          echo ""
          echo "  rm -f /usr/local/bin/set-performance.sh"
          echo "  rm -rf /usr/local/etc/set-performance.conf.d"
          echo "  rm -f /etc/cron.hourly/set-performance"
          echo "  rm -f /etc/cron.daily/set-performance"
          echo "  rm -f /etc/cron.weekly/set-performance"
          echo "  rm -f /etc/cron.monthly/set-performance"
          echo "-----------------------------------------------------------------------"
          echo ""
          read -p "Abort to clean up manually? (Y/n): " choice
          case "$choice" in
            n|N ) echo "Proceeding with caution -- watch for conflicts!"; echo "" ;;
            * )   echo "Installation aborted. Clean up and re-run install.sh."; exit 1 ;;
          esac
      else
          echo "[OK] No legacy components found -- clean install!"
          echo ""
      fi
      
      echo "Creating directory structure..."
      mkdir -p "$INSTALL_BIN/tag-automation/modules"
      mkdir -p "$INSTALL_CONF/tag-automation/conf.d"
      echo "[OK] Directories created"
      
      echo "Installing main.sh..."
      cp "$BUNDLE_DIR/main.sh" "$INSTALL_BIN/tag-automation/main.sh"
      chmod +x "$INSTALL_BIN/tag-automation/main.sh"
      echo "[OK] main.sh installed"
      
      echo "Installing modules..."
      cp "$BUNDLE_DIR/modules/set-performance.sh" \
         "$INSTALL_BIN/tag-automation/modules/set-performance.sh"
      chmod +x "$INSTALL_BIN/tag-automation/modules/set-performance.sh"
      echo "[OK] set-performance.sh installed"
      echo "[--] set-permissions.sh -- Coming Soon (not installed)"
      
      echo "Installing configuration files..."
      cp "$BUNDLE_DIR/conf.d/default.conf" \
         "$INSTALL_CONF/tag-automation/conf.d/default.conf"
      echo "[OK] default.conf installed"
      
      if [ ! -f "$INSTALL_CONF/tag-automation/conf.d/custom.conf" ]; then
          cp "$BUNDLE_DIR/conf.d/custom.conf" \
             "$INSTALL_CONF/tag-automation/conf.d/custom.conf"
          echo "[OK] custom.conf installed (first time)"
      else
          echo "[OK] custom.conf already exists -- skipping (your settings preserved)"
      fi
      
      NFS_CODE_PATH="/mnt/v0/code/tag-automation"
      if [ -d "$NFS_CODE_PATH" ]; then
          mkdir -p "$NFS_CODE_PATH/logs"
          echo "[OK] NFS code path verified : $NFS_CODE_PATH"
      else
          echo "[--] NFS code path not found: $NFS_CODE_PATH"
          echo "     mkdir -p $NFS_CODE_PATH/logs"
      fi
      
      echo ""
      echo "============================================"
      echo "  *** ACTION REQUIRED -- DO NOT SKIP ***"
      echo "============================================"
      echo ""
      echo "  STEP 1: Edit your pool-specific config:"
      echo "    vi $INSTALL_CONF/tag-automation/conf.d/custom.conf"
      echo ""
      echo "    Set your TAG_SUFFIX to match your pool:"
      echo "      POOL-1 --> TAG_SUFFIX=\"-1\""
      echo "      POOL-2 --> TAG_SUFFIX=\"-2\""
      echo "      Single pool --> TAG_SUFFIX=\"\""
      echo ""
      echo "  STEP 2: Initialize the bundle (REQUIRED):"
      echo "    /usr/local/bin/tag-automation/main.sh initialize"
      echo ""
      echo "  STEP 3: Verify it is working:"
      echo "    /usr/local/bin/tag-automation/main.sh"
      echo "    tail /var/log/tag-automation.log"
      echo ""
      echo "  *** Bundle will NOT run until initialize is completed ***"
      echo "============================================"
      

      2-of-5: main.sh

      #!/bin/bash
      
      # ============================================
      # Tag-Based-Automation -- main.sh (v1.0)
      # Orchestrator: loads config, runs modules
      # ============================================
      
      CONF_DIR="/usr/local/etc/tag-automation/conf.d"
      DEFAULT_CONF="$CONF_DIR/default.conf"
      CUSTOM_CONF="$CONF_DIR/custom.conf"
      SCRIPT_PATH="/usr/local/bin/tag-automation/main.sh"
      
      if [ -f "$DEFAULT_CONF" ]; then
          source "$DEFAULT_CONF"
      else
          echo "Error: default.conf not found at $DEFAULT_CONF"
          exit 1
      fi
      [ -f "$CUSTOM_CONF" ] && source "$CUSTOM_CONF"
      
      CORE_TAG="${CORE_BASE}${TAG_SUFFIX}"
      HIGH_TAG="${HIGH_BASE}${TAG_SUFFIX}"
      NORMAL_TAG="${NORMAL_BASE}${TAG_SUFFIX}"
      LOW_TAG="${LOW_BASE}${TAG_SUFFIX}"
      
      initialize_plugin() {
          echo ""
          echo "Tag-Based-Automation -- Initialization"
          echo "============================================"
      
          if [ ! -d "$CONF_DIR" ]; then
              echo "Error: Config directory not found: $CONF_DIR"
              exit 1
          fi
      
          if [ ! -f "$DEFAULT_CONF" ]; then
              echo "Error: default.conf not found."
              exit 1
          fi
      
          [ ! -f "$CUSTOM_CONF" ] && \
              echo "Warning: custom.conf not found. Using defaults only."
      
          TARGET_DIR="/etc/cron.${SCHEDULE}"
          SYMLINK_PATH="${TARGET_DIR}/tag-automation"
      
          if [ ! -d "$TARGET_DIR" ]; then
              echo "Error: Cron directory not found: $TARGET_DIR"
              echo "Valid SCHEDULE options: hourly, daily, weekly, monthly"
              exit 1
          fi
      
          for interval in hourly daily weekly monthly; do
              rm -f "/etc/cron.${interval}/tag-automation"
          done
      
          ln -sf "$SCRIPT_PATH" "$SYMLINK_PATH"
      
          if [ -L "$SYMLINK_PATH" ]; then
              echo "[OK] Symlink created : $SYMLINK_PATH"
              echo "[OK] Schedule        : $SCHEDULE"
              echo "[OK] Tag suffix      : ${TAG_SUFFIX:-(none)}"
              echo "[OK] Active tags     : $CORE_TAG, $HIGH_TAG, $NORMAL_TAG, $LOW_TAG"
              echo "[OK] Performance     : ${ENABLE_PERFORMANCE:-true}"
              echo "[--] Permissions     : Coming Soon"
              echo ""
              echo "Initialization complete. Bundle will run $SCHEDULE via cron."
          else
              echo "Error: Failed to create symlink."
              exit 1
          fi
      }
      
      [ "$1" == "initialize" ] && initialize_plugin && exit 0
      
      # --- MASTER HOST CHECK ---
      POOL_MASTER=$(xe pool-list params=master --minimal)
      LOCAL_HOST=$(xe host-list name-label=$(hostname) --minimal)
      [ "$POOL_MASTER" != "$LOCAL_HOST" ] && exit 0
      
      exec >> "$MAIN_LOG" 2>&1
      echo "--- Tag-Automation Starting: $(date) ---"
      echo "    Tags: $CORE_TAG | $HIGH_TAG | $NORMAL_TAG | $LOW_TAG"
      
      if [ "${ENABLE_PERFORMANCE:-true}" == "true" ]; then
          echo "--- Running set-performance module ---"
          bash /usr/local/bin/tag-automation/modules/set-performance.sh
      fi
      
      # --- set-permissions MODULE (COMING SOON) ---
      # if [ "${ENABLE_PERMISSIONS:-false}" == "true" ]; then
      #     echo "--- Running set-permissions module ---"
      #     bash /usr/local/bin/tag-automation/modules/set-permissions.sh
      # fi
      
      # --- NFS LOG PUSH (graceful -- no auto-remount) ---
      NFS_CODE_PATH="/mnt/v0/code/tag-automation"
      if mountpoint -q "/mnt/v0" && [ -d "$NFS_CODE_PATH" ]; then
          cp "$SUMMARY_LOG" "$NFS_CODE_PATH/logs/" 2>/dev/null
          echo "[OK] Logs pushed to NFS"
      else
          echo "[--] NFS not available -- skipping log push"
      fi
      
      echo "--- Tag-Automation Complete: $(date) ---"
      echo "$(date '+%Y-%m-%d %H:%M:%S') $(hostname) Suffix:${TAG_SUFFIX:-(none)} Performance:${ENABLE_PERFORMANCE:-true}" >> "$SUMMARY_LOG"
      

      3-of-5: modules/set-performance.sh

      #!/bin/bash
      
      # ============================================
      # Tag-Based-Automation Module: set-performance.sh (v3.2)
      # Called by main.sh -- inherits all variables
      # ============================================
      
      count_0=0
      count_1=0
      count_2=0
      count_3=0
      
      echo "--- Starting Performance Sync: $(date) ---"
      echo "    Tags: $CORE_TAG | $HIGH_TAG | $NORMAL_TAG | $LOW_TAG"
      
      echo "=== Applying $CORE_TAG (Weight: $CORE_WEIGHT, Pri: $CORE_IO_PRI) ==="
      for uuid in $(xe vm-list tags:contains="$CORE_TAG" --minimal | tr ',' '\n'); do
          [ -z "$uuid" ] && continue
          xe vm-param-set uuid=$uuid VCPUs-params:weight=$CORE_WEIGHT \
              other-config:sched-pri=$CORE_IO_PRI
          echo "  [OK] CORE applied: $uuid"
          ((count_0++))
      done
      
      echo "=== Applying $HIGH_TAG (Weight: $HIGH_WEIGHT, Pri: $HIGH_IO_PRI) ==="
      for uuid in $(xe vm-list tags:contains="$HIGH_TAG" --minimal | tr ',' '\n'); do
          [ -z "$uuid" ] && continue
          xe vm-param-set uuid=$uuid VCPUs-params:weight=$HIGH_WEIGHT \
              other-config:sched-pri=$HIGH_IO_PRI
          echo "  [OK] HIGH applied: $uuid"
          ((count_1++))
      done
      
      echo "=== Applying $NORMAL_TAG (Weight: $NORMAL_WEIGHT, Pri: $NORMAL_IO_PRI) ==="
      for uuid in $(xe vm-list tags:contains="$NORMAL_TAG" --minimal | tr ',' '\n'); do
          [ -z "$uuid" ] && continue
          xe vm-param-set uuid=$uuid VCPUs-params:weight=$NORMAL_WEIGHT \
              other-config:sched-pri=$NORMAL_IO_PRI
          echo "  [OK] NORMAL applied: $uuid"
          ((count_2++))
      done
      
      echo "=== Applying $LOW_TAG (Weight: $LOW_WEIGHT, Pri: $LOW_IO_PRI) ==="
      for uuid in $(xe vm-list tags:contains="$LOW_TAG" --minimal | tr ',' '\n'); do
          [ -z "$uuid" ] && continue
          xe vm-param-set uuid=$uuid VCPUs-params:weight=$LOW_WEIGHT \
              other-config:sched-pri=$LOW_IO_PRI
          echo "  [OK] LOW applied: $uuid"
          ((count_3++))
      done
      
      echo "--- Performance Sync Complete: $(date) ---"
      echo "$(date '+%Y-%m-%d %H:%M:%S') $(hostname) \
          $CORE_TAG:$count_0 $HIGH_TAG:$count_1 \
          $NORMAL_TAG:$count_2 $LOW_TAG:$count_3" >> "$SUMMARY_LOG"
      

      4-of-5: conf.d/default.conf

      # ============================================
      # default.conf
      # Global defaults for Tag-Based-Automation
      # DO NOT edit -- use custom.conf for overrides
      # ============================================
      
      SCHEDULE="hourly"
      
      ENABLE_PERFORMANCE=true
      # ENABLE_PERMISSIONS=false  # Coming Soon
      
      MAIN_LOG="/var/log/tag-automation.log"
      SUMMARY_LOG="/var/log/tag-automation-summary.log"
      
      CORE_BASE="0-core"
      HIGH_BASE="1-high"
      NORMAL_BASE="2-normal"
      LOW_BASE="3-low"
      
      CORE_WEIGHT="2048"
      CORE_IO_PRI="7"
      
      HIGH_WEIGHT="1024"
      HIGH_IO_PRI="7"
      
      NORMAL_WEIGHT="256"
      NORMAL_IO_PRI="4"
      
      LOW_WEIGHT="128"
      LOW_IO_PRI="1"
      

      5-of-5: conf.d/custom.conf

      # ============================================
      # custom.conf
      # Pool-specific overrides
      # Edit this file to customize your environment
      # ============================================
      
      # --- TAG SUFFIX ---
      # POOL-1 --> TAG_SUFFIX="-1"
      # POOL-2 --> TAG_SUFFIX="-2"
      # Generic --> TAG_SUFFIX=""
      TAG_SUFFIX=""
      
      # --- OPTIONAL OVERRIDES ---
      # Uncomment to override default.conf values:
      # SCHEDULE="daily"
      # ENABLE_PERFORMANCE=true
      # CORE_WEIGHT="4096"
      # HIGH_WEIGHT="2048"
      # NORMAL_WEIGHT="512"
      # LOW_WEIGHT="256"
      
      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

      @poddingue Thank you!
      "Anything and Everything we can to to improve XCP-ng", is the "Name of the Game" 🙂

      posted in Hardware
      johnnezeroJ
      johnnezero

    Latest posts made by johnnezero

    • RE: Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      @julienXOvates Noted, thank you!

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      @MathieuRA Wow, and wow! Thanks for the detail on RBAC, I'll have give it a look. Unfortunately the current plugin doesn't get any where near the "deep-weeds" of all that, and basically just uses the built-in "Admin, Operator and Viewer" roles to auto-provision (via Autopilot) new VMs as they show up (i.e. during a migration project, etc. ). We needed something for our current VMware-to-XCP-ng migration project, and '"Where's there's a will, there's a way" kicked in and the plugin came to be.
      Thanks again for your input, and let me know if anything else comes to mind...
      Happy Day 🙂

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      @poddingue FYI: I posted the "Feature Request" to the feedback channel here as you suggested.
      Hopefully if enough people were used to the way VMware allowed "multiple granular permissions to be set per VM", it will get some traction. Thanks again for pointing me in the right direction.
      Happy Day 🙂

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      @poddingue Thank you for the feedback, and suggestion. I will post a request regarding the one-ACL-per-VM issue, as I know it's an issue with our current 600+ VM VMware-to-XCP-ng migration project. Happy Day 🙂

      posted in Management
      johnnezeroJ
      johnnezero
    • Tag-Based Automation Plugin: Tag-Based VM Performance & Permission Management via assigned tag(s)

      UPDATE: Tag-Based-Automation Plugin

      WHAT: The xo-tag-automation plugin is a native xo-server plugin that enforces VM performance tiers, manages group permissions, and synchronizes VM metadata -- all driven by VM tags and NFS-hosted CSV files (see below for more details). In general only apply one performance and/or permission action tag per VM. Note: Currently XO only allows one ACL assignment per VM (e.g. assigning "Dept1-Operator", and" Dept3-Admin" to the same VM does not work).

      So brace yourself as there's a lot to chew below,, but I promise if your actively involved with a sizable migration project like we are (a.k.a. "Who can afford VMware anymore?"), it will be well worth your time once you have the plugin setup and running (especially in the "Set-it and Forget-it" Permissions AutoPiolot mode). Enjoy....

      FEATURES


      PERFORMANCE SYNC
      Automatically applies CPU weights and IO priorities (sched-pri) based on VM performance tier tags:

      0-core   -- CPU weight: 2048  IO priority: 7
      1-high   -- CPU weight: 1024  IO priority: 7
      2-normal -- CPU weight: 512   IO priority: 5
      3-low    -- CPU weight: 256   IO priority: 2
      

      All tier weights and IO priorities are fully configurable in the plugin UI. An optional pool-specific tag suffix (e.g. -1=POOL1, -2=POOL2, etc.) allows you to manage multiple pools from a single XO instance without tag collisions.
      Note: Under normal conditions these performance settings do not affect anything. It's only when system resources are under high-load that the tiering levels ensure important VMs get priority over non-critical workloads, etc.


      PERMISSION SYNC
      VM tags ending in -Admin, -Operator, or -Viewer (e.g. "Dept1-Operator", "Dept2-Admin", etc.) automatically trigger appropriate XO Group creation and ACL assignments for the designated VM.

      CSV PERMISSION MANAGEMENT FILES
      Two CSV files on your NFS share drive the workflow:

      current-vms.csv
      A live export of your entire VM inventory. Contains UUID, Name, CurrentTags, NewTags, CurrentNotes, and NewNotes columns. Edit the NewTags and NewNotes columns and run the plugin (e.g. push the [Test plugin] button) to apply changes in bulk. The CSV auto-refreshes after each run. A staleness warning fires if the CSV has not been updated
      within a configurable number of days.

      preload-vms.csv
      Pre-stage tag and notes configurations for VMs that do not exist yet -- before they are migrated or created. The moment a VM matching a preload entry appears in XO, the plugin applies (on next scheduled interval) its tags and notes automatically and removes the entry from the preload file.


      PERMISSION AUTOPILOT
      Designed for active migration and onboarding projects. Automatically applies permission settings on scheduled interval, based on preload-vms.csv contents. Note: Should be disabled when not actively involved in migration projects.


      DRY-RUN / EXPORT-CSV MODE
      When Dry-Run is ON (the default), the plugin previews all changes in the XO logs without applying any changes, and simultaneously exports a fresh copy of all VM metadata to current-vms.csv with blank NewTags and NewNotes columns ready to fill in. Turn Dry-Run OFF to apply changes for real.


      RUN NOW (e.g. The [Test plugin] button)
      Trigger a full enforcement cycle instantly from the XO plugin UI without waiting for the next scheduled run.


      NFS LOGGING
      All activity is written to structured log files on your NFS share:

      xo-tag-automation.log         -- full run log (auto-rotates at 2MB)
      xo-tag-automation.log.1      -- previous log backup
      xo-tag-automation-summary.log -- run summary entries only
      daily-summary.log             -- nightly VM count and new VM report
      

      Logs are also available via xo-cli API methods (see below).


      LEGACY MIGRATION
      If you have an existing vm_metadata.csv from an older version, the plugin automatically renames it to current-vms.csv on first run. No manual migration needed.


      SECURITY -- PLEASE READ CAREFULLY

      This plugin automates infrastructure changes. Security is not optional. Please take both of the following seriously before enabling Permission Sync or Autopilot.


      -- REST API SERVICE ACCOUNT --

      This plugin uses the XO JSON-RPC API internally. Use a dedicated service account (never your personal admin credentials). Note: REST API is only required for the Permissions Sync functions, it's not required if you just want to use the Performance Sync plugin functionality.

      Recommended setup (generic -- adapt to your environment):

      1. Create a dedicated XO user account for the service (e.g. a non-admin account with only the minimal access permissions required)

      2. Generate a scoped API token via xo-cli:
        xo-cli --register
        xo-cli token.create

      3. Store the token securely -- treat it like a password!

      4. Refer to the official Vates REST API documentation for full token management guidance:
        https://xen-orchestra.com/docs/restapi.html


      -- NFS SHARE SECURITY --

      The NFS share hosts your CSV files and logs. Anyone with write access to the share can modify VM tags and permissions.

      STRONGLY RECOMMENDED

      • Run the NFS share from a dedicated VM -- not a general-purpose NFS server.

      • Restrict NFS exports to the XOA IP address only:
        /srv/nfs/share<XOAIP>/32(rw,sync,no_subtree_check,no_root_squash)

      • Do NOT expose the NFS share to the general network or to end-user access.

      • Admins who need to edit CSV files can SCP them to/from the XOA:
        EXAMPLE:
        Download CSV from XOA to your workstation
        scp <xoa-user>@<xoa-ip>:/path/to/current-vms.csv ./

        Upload edited CSV back to XOA
        scp ./current-vms.csv <xoa-user>@<xoa-ip>:/path/to/current-vms.csv

      • Use firewall rules to enforce NFS access at the network level in addition to the exports configuration

      Failure to secure the NFS share is a serious security risk.


      UPGRADING FROM THE STANDALONE SCRIPT

      If you installed the old standalone set-performance.sh script, you can remove it before enabling the plugin as follows:

      1. Remove the script:

        sudo rm /usr/local/bin/set-performance.sh
        
      2. Remove the crontab entry:

          crontab -e
          (delete the line referencing set-performance.sh)
        
      3. Note: The plugin handles its own scheduling via the XO UI. No manual cron configuration is required.


      INSTALLATION

      1. Download the latest airgap release tarball from GitHub:
        https://github.com/johnezero/xo-tag-automation_plugin/releases

      2. SCP the tarball to your XOA:

        scp xo-tag-automation-airgap-vX.X.X.tar.gz <xoa-user>@<xoa-ip>:/tmp/
        
      3. Create the plugin folder:

        sudo mkdir -p /usr/local/lib/node_modules/xo-server-tag-automation
        
      4. Extract directly into the plugin directory:

        sudo tar -xzvf /tmp/xo-tag-automation-airgap-vX.X.X.tar.gz -C /usr/local/lib/node_modules/xo-server-tag-automation/ --strip-components=1
        
      5. Restart xo-server:

        sudo systemctl restart xo-server
        
      6. Verify registration:

        sudo journalctl -u xo-server -n 100 --no-pager | grep -A3 "tag-automation"
        

        You should see:

        [INFO] xo-tag-automation: Plugin factory called -- xo context: YES
        [INFO] xo-tag-automation: Plugin loaded -- waiting for core started.
        xo:plugin INFO successfully register tag-automation
        
      7. Enable and configure the plugin options in XO:
        Settings -> Plugins -> tag-automation -> Enable


      XO-CLI API METHODS

      The plugin exposes several API methods accessible via xo-cli:

      xo-cli xo-server-tag-automation.exportCsv
      Export current VM inventory to current-vms.csv

      xo-cli xo-server-tag-automation.downloadCsvApi
      Print CSV content to stdout

      xo-cli xo-server-tag-automation.uploadCsvApi content@./current-vms.csv
      Push an edited CSV back to the NFS share

      xo-cli xo-server-tag-automation.getLog lines=100
      View the last N lines of the plugin log

      xo-cli xo-server-tag-automation.getDailySummary
      View the nightly VM count summary

      xo-cli xo-server-tag-automation.getFilePaths
      Show all configured file paths


      LINKS

      GitHub: https://github.com/johnezero/xo-tag-automation_plugin
      Vates REST API docs: https://xen-orchestra.com/docs/restapi.html


      STANDARD DISCLAIMER HERE

      This software is provided "AS-IS" without any express or implied warranty. While this plugin is being used in a production environment managing live VMs, you should always review the code and test it in a non-production environment before full deployment.
      Note: The plugin is designed to ONLY take action on VMs with specific tags assigned (e.g. untagged VMs are never modified).
      Use Dry-Run mode to preview all changes (via log file output) before applying them.

      Feedback, input and suggestions and always welcome...

      johnezero-logo.jpg

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

      @poddingue Thank you!
      "Anything and Everything we can to to improve XCP-ng", is the "Name of the Game" 🙂

      posted in Hardware
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      @tjkreidl Yes, it's all good! Happy Day 🙂

      posted in Management
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      @tjkreidl Found it. I re-posted the topic back to the hardware category (and linked the pdf's to the #tobiaskreidl github) here:
      Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

      posted in Management
      johnnezeroJ
      johnnezero
    • Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

      WHAT: This post is dedicated to preserving the valuable server administration guide content produced by @tjkreidl, originally titled "A Tale of Two Servers." These articles were first published in 2019 on the Citrix Blogs, which are unfortunately no longer available.

      KUDOS: Special thanks to @john.c for performing some incredible web sleuthing to recover this content, and to @tjkreidl for creating these essential guides in the first place — as well as for "Breathing Life" back into them by highlighting their importance before they disappeared forever into the ferocious "Bit-Bucket."

      Series Overview & Quick Reference

      If you're looking for a high-level summary of the concepts covered in this series — including specific XCP-ng/XO commands for BIOS power management, GPU scheduling, and NUMA inspection — start here:
      Quick Reference & Summary Guide

      The Complete Series

      =========================================

      Part 1: How BIOS Settings Can Affect Your Apps and GPU Performance:

      A deep dive into CPU Power Management (OS DBPM vs. System DBPM) and Turbo mode.
      Read Article

      Part 2: GPU Settings and Advanced BIOS Tuning

      Exploring Uncore frequency, C1E states, and NVIDIA GPU Scheduler modes (Best Effort vs. Equal Share).
      Read Article

      Part 3: NUMA, CPUs, Sockets/Cores, and VM Performance

      Understanding vNUMA, vCPU oversubscription, and the importance of VM startup order for optimal memory placement.
      Read Article

      Archived for the XCP-ng community. Questions or additional insights? Please discuss below!

      posted in Hardware
      johnnezeroJ
      johnnezero
    • RE: Tag-Based Automation: Manage VM CPU Priority via assigned tag.

      @john.c Will do - and kudos to you on some phenomenal sleuthing work (and to @tjkreidl for sparking ithis contents recall from an early "bit-bucket" grave)! Happy Day All!

      posted in Management
      johnnezeroJ
      johnnezero