@poddingue Thank you!
"Anything and Everything we can to to improve XCP-ng", is the "Name of the Game"

@tjkreidl Yes, it's all good! Happy Day

@tjkreidl Found it. I re-posted the topic back to the hardware category (and linked the pdf's to the #tobiaskreidl github) here:
Server Admin Guide: A Tale of Two Servers: BIOS, GPU, and NUMA Tuning for XCP-ng: Preserving the valuable work done by Tobias Kreidl (@tjkreidl)

WHAT: This post is dedicated to preserving the valuable server administration guide content produced by @tjkreidl, originally titled "A Tale of Two Servers." These articles were first published in 2019 on the Citrix Blogs, which are unfortunately no longer available.

KUDOS: Special thanks to @john.c for performing some incredible web sleuthing to recover this content, and to @tjkreidl for creating these essential guides in the first place — as well as for "Breathing Life" back into them by highlighting their importance before they disappeared forever into the ferocious "Bit-Bucket."

Series Overview & Quick Reference

If you're looking for a high-level summary of the concepts covered in this series — including specific XCP-ng/XO commands for BIOS power management, GPU scheduling, and NUMA inspection — start here:
Quick Reference & Summary Guide

The Complete Series

=========================================

Part 1: How BIOS Settings Can Affect Your Apps and GPU Performance:

A deep dive into CPU Power Management (OS DBPM vs. System DBPM) and Turbo mode.
Read Article

Part 2: GPU Settings and Advanced BIOS Tuning

Exploring Uncore frequency, C1E states, and NVIDIA GPU Scheduler modes (Best Effort vs. Equal Share).
Read Article

Part 3: NUMA, CPUs, Sockets/Cores, and VM Performance

Understanding vNUMA, vCPU oversubscription, and the importance of VM startup order for optimal memory placement.
Read Article

Archived for the XCP-ng community. Questions or additional insights? Please discuss below!

@john.c Will do - and kudos to you on some phenomenal sleuthing work (and to @tjkreidl for sparking ithis contents recall from an early "bit-bucket" grave)! Happy Day All!

@john.c FYI: Bundling/done (Plugin/To come...)

@tjkreidl Wow that all sounds pretty intense - will keep it in mind. Thanks much!

@john.c Thanks much, looking into it.
"Open-Source for the Win!"

@Pilow Sounds like an awesome idea. Send any details you may have on how to make plugins (if you know how that is).
Adding to the ToDo list - Thanks!

@john.c Also done! Thanks for all the great input, keep em' coming...

@john.c Wandered off through the weeds (with Claude/AI that us), and got it done.

@john.c Yet another awesome idea - adding it to the "ToDo List", thanks!

@john.c Great ideas (especially if Vates decides to bake something similar into XO someday) but may be getting too far into the weeds for now...

@john.c Great idea, and done! ("Keep 'em coming")

@Danp Thanks for the correction - i.e. "Good-eye, Good-eye

UPDATE: Tag-Based-Automation-Bundle v1.0

WHAT'S NEW: :This update introduces the "Tag-Based-Automation" bundle -- a modular, install.sh-based package that replaces the original standalone set-performance.sh script.

This bundle is the foundation for the upcoming "Tag-Based-Automation" plugin for Xen Orchestra (if/when all that gets worked out with Vates).
For now it is a production-ready shell-based bundle that installs cleanly on all XCP-ng pool hosts, and will only run on the current Pool Master.

The goal is simple: Metadata (Tags) should drive Infrastructure State.

STANDARD DISCLAIMER HERE

This software is provided "AS-IS" without any express or implied warranty. While these scripts are being used in a production environment managing VMs, you should always review the code and test it in a non-production environment before full deployment.
Note: The scripts are designed to ONLY take action on VMs with specific tags assigned -- untagged VMs are never touched.
That said, as always - your mileage may vary...

WHAT IS IN THE BUNDLE?

[ACTIVE]      set-performance.sh
              Automated enforcement of CPU weights and I/O priorities
              based on assigned VM tags. Runs via cron. Prevents
              configuration drift across your entire pool.

[COMING SOON] Manage Tags via CSV
              Bulk VM metadata management using a CSV as Source of
              Truth workflow. Export your VM list, edit tags in Excel,
              sync back to the pool in one command.

[COMING SOON] set-permissions.sh
              Automated management of Xen Orchestra Resource Sets and
              user permissions -- driven entirely by your existing VM
              tags. No manual Resource Set updates ever again.

CHANGE-LOG

v1.0 - 2026-05-16 (Bundle Release)
  - Modular Architecture: Repackaged as a modular bundle for easier
    updates and future plugin development
  - One-Shot Installer: install.sh handles directory creation,
    permissions, and legacy standalone cleanup detection
  - Orchestrator: main.sh manages module execution and cron
    scheduling via symlink -- no manual crontab editing required
  - Dual Config System: default.conf (never edit) + custom.conf
    (your pool settings) -- your settings are preserved on re-install
  - NFS Integration: Optional log aggregation for centralized
    reporting across multiple pools
  - Master Host Check: Robust UUID comparison ensures script only
    runs on the Pool Master -- safe to deploy on all hosts
  - Foundation for upcoming Xen Orchestra plugin

v3.2 - 2026-05-14 (Final Standalone Release)
  - Fixed Master Host Check: Replaced invalid xe host-is-master
    with correct UUID comparison method

v3.1 - 2026-05-14
  - Config Separation: Moved to conf.d/ override pattern
  - Split into default.conf and custom.conf

v3.0 - 2026-05-14
  - TAG_SUFFIX support for multi-pool deployments
  - Auto-scheduling via initialize -- no manual crontab needed

v2.0 - 2026-05-14
  - External configuration file
  - Master Host Check (first attempt)
  - Main log + summary log
  - Per-tier summary counters

v1.0 - Initial Release
  - Tag-based CPU weight and I/O priority enforcement
  - Four tiers: 0-core, 1-high, 2-normal, 3-low
  - Network QoS cap (100Mbps) for 3-low tagged VMs

CLEANUP: REMOVING THE OLD STANDALONE VERSION (If installed)

If you installed the original standalone script, run these first:

  rm -f /usr/local/bin/set-performance.sh
  rm -rf /usr/local/etc/set-performance.conf.d
  rm -f /etc/cron.hourly/set-performance
  rm -f /etc/cron.daily/set-performance
  rm -f /etc/cron.weekly/set-performance
  rm -f /etc/cron.monthly/set-performance

DOWNLOAD (Recommended)

Download the v1.0 bundle directly from GitHub:
https://github.com/johnezero/xo-tag-automation/releases/download/v1.0/tag-automation-v1.0.tar.gz

INSTALLING THE BUNDLE

STEP 1: Download and extract on your Pool Master

wget https://github.com/johnezero/xo-tag-automation/releases/download/v1.0/tag-automation-v1.0.tar.gz
tar -xzvf tag-automation-v1.0.tar.gz
cd tag-automation-bundle/

-- OR --

Download to your workstation and upload via SCP or WinSCP:
scp tag-automation-v1.0.tar.gz root@your-pool-master:/root/
Then on your Pool Master:
tar -xzvf tag-automation-v1.0.tar.gz
cd tag-automation-bundle/

STEP 2: Run the installer
chmod +x install.sh
./install.sh

The installer will:
- Detect and warn about any legacy standalone components
- Create all required directories
- Deploy all scripts and config files
- Preserve any existing custom.conf on re-install
- Verify your NFS code path if applicable

STEP 3: Edit your pool-specific config:
*** ACTION REQUIRED -- DO NOT SKIP ***
vi /usr/local/etc/tag-automation/conf.d/custom.conf

Set your TAG_SUFFIX to match your pool:
POOL-1 --> TAG_SUFFIX="-1"
POOL-2 --> TAG_SUFFIX="-2"
Single pool --> TAG_SUFFIX=""

STEP 4: Initialize the bundle (REQUIRED):
*** ACTION REQUIRED -- DO NOT SKIP ***
/usr/local/bin/tag-automation/main.sh initialize

Expected output:
[OK] Symlink created : /etc/cron.hourly/tag-automation
[OK] Schedule : hourly
[OK] Tag suffix : (none)
[OK] Active tags : 0-core, 1-high, 2-normal, 3-low
[OK] Performance : true
[--] Permissions : Coming Soon

*** Bundle will NOT run until initialize is completed ***

STEP 5: Verify it is working
/usr/local/bin/tag-automation/main.sh
tail /var/log/tag-automation.log
tail /var/log/tag-automation-summary.log

THE FILES (i.e. The hard way)

1-of-5: install.sh
Note: The install bundle can e located anywhere (i.e. /root/tag-automation-bundle)

--------------------------------------------

#!/bin/bash

# ============================================
# Tag-Based-Automation -- install.sh (v1.1)
# One-shot installer for the full bundle
# ============================================

BUNDLE_DIR="$(dirname "$0")"
INSTALL_BIN="/usr/local/bin"
INSTALL_CONF="/usr/local/etc"

echo ""
echo "Tag-Based-Automation -- Installer v1.1"
echo "============================================"
echo ""

if [ "$(id -u)" != "0" ]; then
    echo "Error: This installer must be run as root."
    exit 1
fi

if ! command -v xe &>/dev/null; then
    echo "Error: xe command not found."
    echo "This installer must be run on an XCP-ng Pool Master."
    exit 1
fi

echo "Checking for legacy standalone components..."
LEGACY_FOUND=false

[ -f "/usr/local/bin/set-performance.sh" ] && \
    echo "  [!] Legacy script found: /usr/local/bin/set-performance.sh" && \
    LEGACY_FOUND=true

[ -d "/usr/local/etc/set-performance.conf.d" ] && \
    echo "  [!] Legacy config dir found: /usr/local/etc/set-performance.conf.d" && \
    LEGACY_FOUND=true

crontab -l 2>/dev/null | grep -q "set-performance.sh" && \
    echo "  [!] Legacy crontab entry detected." && \
    LEGACY_FOUND=true

LEGACY_CRONS=$(find /etc/cron.hourly /etc/cron.daily /etc/cron.weekly \
    /etc/cron.monthly -name "set-performance" 2>/dev/null)
if [ -n "$LEGACY_CRONS" ]; then
    echo "  [!] Legacy cron symlinks detected:"
    echo "$LEGACY_CRONS" | sed 's/^/      - /'
    LEGACY_FOUND=true
fi

if [ "$LEGACY_FOUND" = true ]; then
    echo ""
    echo "-----------------------------------------------------------------------"
    echo "CAUTION: Legacy standalone components detected."
    echo "Please remove them before proceeding:"
    echo ""
    echo "  rm -f /usr/local/bin/set-performance.sh"
    echo "  rm -rf /usr/local/etc/set-performance.conf.d"
    echo "  rm -f /etc/cron.hourly/set-performance"
    echo "  rm -f /etc/cron.daily/set-performance"
    echo "  rm -f /etc/cron.weekly/set-performance"
    echo "  rm -f /etc/cron.monthly/set-performance"
    echo "-----------------------------------------------------------------------"
    echo ""
    read -p "Abort to clean up manually? (Y/n): " choice
    case "$choice" in
      n|N ) echo "Proceeding with caution -- watch for conflicts!"; echo "" ;;
      * )   echo "Installation aborted. Clean up and re-run install.sh."; exit 1 ;;
    esac
else
    echo "[OK] No legacy components found -- clean install!"
    echo ""
fi

echo "Creating directory structure..."
mkdir -p "$INSTALL_BIN/tag-automation/modules"
mkdir -p "$INSTALL_CONF/tag-automation/conf.d"
echo "[OK] Directories created"

echo "Installing main.sh..."
cp "$BUNDLE_DIR/main.sh" "$INSTALL_BIN/tag-automation/main.sh"
chmod +x "$INSTALL_BIN/tag-automation/main.sh"
echo "[OK] main.sh installed"

echo "Installing modules..."
cp "$BUNDLE_DIR/modules/set-performance.sh" \
   "$INSTALL_BIN/tag-automation/modules/set-performance.sh"
chmod +x "$INSTALL_BIN/tag-automation/modules/set-performance.sh"
echo "[OK] set-performance.sh installed"
echo "[--] set-permissions.sh -- Coming Soon (not installed)"

echo "Installing configuration files..."
cp "$BUNDLE_DIR/conf.d/default.conf" \
   "$INSTALL_CONF/tag-automation/conf.d/default.conf"
echo "[OK] default.conf installed"

if [ ! -f "$INSTALL_CONF/tag-automation/conf.d/custom.conf" ]; then
    cp "$BUNDLE_DIR/conf.d/custom.conf" \
       "$INSTALL_CONF/tag-automation/conf.d/custom.conf"
    echo "[OK] custom.conf installed (first time)"
else
    echo "[OK] custom.conf already exists -- skipping (your settings preserved)"
fi

NFS_CODE_PATH="/mnt/v0/code/tag-automation"
if [ -d "$NFS_CODE_PATH" ]; then
    mkdir -p "$NFS_CODE_PATH/logs"
    echo "[OK] NFS code path verified : $NFS_CODE_PATH"
else
    echo "[--] NFS code path not found: $NFS_CODE_PATH"
    echo "     mkdir -p $NFS_CODE_PATH/logs"
fi

echo ""
echo "============================================"
echo "  *** ACTION REQUIRED -- DO NOT SKIP ***"
echo "============================================"
echo ""
echo "  STEP 1: Edit your pool-specific config:"
echo "    vi $INSTALL_CONF/tag-automation/conf.d/custom.conf"
echo ""
echo "    Set your TAG_SUFFIX to match your pool:"
echo "      POOL-1 --> TAG_SUFFIX=\"-1\""
echo "      POOL-2 --> TAG_SUFFIX=\"-2\""
echo "      Single pool --> TAG_SUFFIX=\"\""
echo ""
echo "  STEP 2: Initialize the bundle (REQUIRED):"
echo "    /usr/local/bin/tag-automation/main.sh initialize"
echo ""
echo "  STEP 3: Verify it is working:"
echo "    /usr/local/bin/tag-automation/main.sh"
echo "    tail /var/log/tag-automation.log"
echo ""
echo "  *** Bundle will NOT run until initialize is completed ***"
echo "============================================"

2-of-5: main.sh

#!/bin/bash

# ============================================
# Tag-Based-Automation -- main.sh (v1.0)
# Orchestrator: loads config, runs modules
# ============================================

CONF_DIR="/usr/local/etc/tag-automation/conf.d"
DEFAULT_CONF="$CONF_DIR/default.conf"
CUSTOM_CONF="$CONF_DIR/custom.conf"
SCRIPT_PATH="/usr/local/bin/tag-automation/main.sh"

if [ -f "$DEFAULT_CONF" ]; then
    source "$DEFAULT_CONF"
else
    echo "Error: default.conf not found at $DEFAULT_CONF"
    exit 1
fi
[ -f "$CUSTOM_CONF" ] && source "$CUSTOM_CONF"

CORE_TAG="${CORE_BASE}${TAG_SUFFIX}"
HIGH_TAG="${HIGH_BASE}${TAG_SUFFIX}"
NORMAL_TAG="${NORMAL_BASE}${TAG_SUFFIX}"
LOW_TAG="${LOW_BASE}${TAG_SUFFIX}"

initialize_plugin() {
    echo ""
    echo "Tag-Based-Automation -- Initialization"
    echo "============================================"

    if [ ! -d "$CONF_DIR" ]; then
        echo "Error: Config directory not found: $CONF_DIR"
        exit 1
    fi

    if [ ! -f "$DEFAULT_CONF" ]; then
        echo "Error: default.conf not found."
        exit 1
    fi

    [ ! -f "$CUSTOM_CONF" ] && \
        echo "Warning: custom.conf not found. Using defaults only."

    TARGET_DIR="/etc/cron.${SCHEDULE}"
    SYMLINK_PATH="${TARGET_DIR}/tag-automation"

    if [ ! -d "$TARGET_DIR" ]; then
        echo "Error: Cron directory not found: $TARGET_DIR"
        echo "Valid SCHEDULE options: hourly, daily, weekly, monthly"
        exit 1
    fi

    for interval in hourly daily weekly monthly; do
        rm -f "/etc/cron.${interval}/tag-automation"
    done

    ln -sf "$SCRIPT_PATH" "$SYMLINK_PATH"

    if [ -L "$SYMLINK_PATH" ]; then
        echo "[OK] Symlink created : $SYMLINK_PATH"
        echo "[OK] Schedule        : $SCHEDULE"
        echo "[OK] Tag suffix      : ${TAG_SUFFIX:-(none)}"
        echo "[OK] Active tags     : $CORE_TAG, $HIGH_TAG, $NORMAL_TAG, $LOW_TAG"
        echo "[OK] Performance     : ${ENABLE_PERFORMANCE:-true}"
        echo "[--] Permissions     : Coming Soon"
        echo ""
        echo "Initialization complete. Bundle will run $SCHEDULE via cron."
    else
        echo "Error: Failed to create symlink."
        exit 1
    fi
}

[ "$1" == "initialize" ] && initialize_plugin && exit 0

# --- MASTER HOST CHECK ---
POOL_MASTER=$(xe pool-list params=master --minimal)
LOCAL_HOST=$(xe host-list name-label=$(hostname) --minimal)
[ "$POOL_MASTER" != "$LOCAL_HOST" ] && exit 0

exec >> "$MAIN_LOG" 2>&1
echo "--- Tag-Automation Starting: $(date) ---"
echo "    Tags: $CORE_TAG | $HIGH_TAG | $NORMAL_TAG | $LOW_TAG"

if [ "${ENABLE_PERFORMANCE:-true}" == "true" ]; then
    echo "--- Running set-performance module ---"
    bash /usr/local/bin/tag-automation/modules/set-performance.sh
fi

# --- set-permissions MODULE (COMING SOON) ---
# if [ "${ENABLE_PERMISSIONS:-false}" == "true" ]; then
#     echo "--- Running set-permissions module ---"
#     bash /usr/local/bin/tag-automation/modules/set-permissions.sh
# fi

# --- NFS LOG PUSH (graceful -- no auto-remount) ---
NFS_CODE_PATH="/mnt/v0/code/tag-automation"
if mountpoint -q "/mnt/v0" && [ -d "$NFS_CODE_PATH" ]; then
    cp "$SUMMARY_LOG" "$NFS_CODE_PATH/logs/" 2>/dev/null
    echo "[OK] Logs pushed to NFS"
else
    echo "[--] NFS not available -- skipping log push"
fi

echo "--- Tag-Automation Complete: $(date) ---"
echo "$(date '+%Y-%m-%d %H:%M:%S') $(hostname) Suffix:${TAG_SUFFIX:-(none)} Performance:${ENABLE_PERFORMANCE:-true}" >> "$SUMMARY_LOG"

3-of-5: modules/set-performance.sh

#!/bin/bash

# ============================================
# Tag-Based-Automation Module: set-performance.sh (v3.2)
# Called by main.sh -- inherits all variables
# ============================================

count_0=0
count_1=0
count_2=0
count_3=0

echo "--- Starting Performance Sync: $(date) ---"
echo "    Tags: $CORE_TAG | $HIGH_TAG | $NORMAL_TAG | $LOW_TAG"

echo "=== Applying $CORE_TAG (Weight: $CORE_WEIGHT, Pri: $CORE_IO_PRI) ==="
for uuid in $(xe vm-list tags:contains="$CORE_TAG" --minimal | tr ',' '\n'); do
    [ -z "$uuid" ] && continue
    xe vm-param-set uuid=$uuid VCPUs-params:weight=$CORE_WEIGHT \
        other-config:sched-pri=$CORE_IO_PRI
    echo "  [OK] CORE applied: $uuid"
    ((count_0++))
done

echo "=== Applying $HIGH_TAG (Weight: $HIGH_WEIGHT, Pri: $HIGH_IO_PRI) ==="
for uuid in $(xe vm-list tags:contains="$HIGH_TAG" --minimal | tr ',' '\n'); do
    [ -z "$uuid" ] && continue
    xe vm-param-set uuid=$uuid VCPUs-params:weight=$HIGH_WEIGHT \
        other-config:sched-pri=$HIGH_IO_PRI
    echo "  [OK] HIGH applied: $uuid"
    ((count_1++))
done

echo "=== Applying $NORMAL_TAG (Weight: $NORMAL_WEIGHT, Pri: $NORMAL_IO_PRI) ==="
for uuid in $(xe vm-list tags:contains="$NORMAL_TAG" --minimal | tr ',' '\n'); do
    [ -z "$uuid" ] && continue
    xe vm-param-set uuid=$uuid VCPUs-params:weight=$NORMAL_WEIGHT \
        other-config:sched-pri=$NORMAL_IO_PRI
    echo "  [OK] NORMAL applied: $uuid"
    ((count_2++))
done

echo "=== Applying $LOW_TAG (Weight: $LOW_WEIGHT, Pri: $LOW_IO_PRI) ==="
for uuid in $(xe vm-list tags:contains="$LOW_TAG" --minimal | tr ',' '\n'); do
    [ -z "$uuid" ] && continue
    xe vm-param-set uuid=$uuid VCPUs-params:weight=$LOW_WEIGHT \
        other-config:sched-pri=$LOW_IO_PRI
    echo "  [OK] LOW applied: $uuid"
    ((count_3++))
done

echo "--- Performance Sync Complete: $(date) ---"
echo "$(date '+%Y-%m-%d %H:%M:%S') $(hostname) \
    $CORE_TAG:$count_0 $HIGH_TAG:$count_1 \
    $NORMAL_TAG:$count_2 $LOW_TAG:$count_3" >> "$SUMMARY_LOG"

4-of-5: conf.d/default.conf

# ============================================
# default.conf
# Global defaults for Tag-Based-Automation
# DO NOT edit -- use custom.conf for overrides
# ============================================

SCHEDULE="hourly"

ENABLE_PERFORMANCE=true
# ENABLE_PERMISSIONS=false  # Coming Soon

MAIN_LOG="/var/log/tag-automation.log"
SUMMARY_LOG="/var/log/tag-automation-summary.log"

CORE_BASE="0-core"
HIGH_BASE="1-high"
NORMAL_BASE="2-normal"
LOW_BASE="3-low"

CORE_WEIGHT="2048"
CORE_IO_PRI="7"

HIGH_WEIGHT="1024"
HIGH_IO_PRI="7"

NORMAL_WEIGHT="256"
NORMAL_IO_PRI="4"

LOW_WEIGHT="128"
LOW_IO_PRI="1"

5-of-5: conf.d/custom.conf

# ============================================
# custom.conf
# Pool-specific overrides
# Edit this file to customize your environment
# ============================================

# --- TAG SUFFIX ---
# POOL-1 --> TAG_SUFFIX="-1"
# POOL-2 --> TAG_SUFFIX="-2"
# Generic --> TAG_SUFFIX=""
TAG_SUFFIX=""

# --- OPTIONAL OVERRIDES ---
# Uncomment to override default.conf values:
# SCHEDULE="daily"
# ENABLE_PERFORMANCE=true
# CORE_WEIGHT="4096"
# HIGH_WEIGHT="2048"
# NORMAL_WEIGHT="512"
# LOW_WEIGHT="256"