Offline
I do read the Blogs. They're one of the better/best blogs out there 
I guess I don't (didn't) understand LTS vs updates vs upgrades.
System is airgapped so Xen Orchestra has no idea there are updates
My long-running goal is to build an edge server VM to clone repos for updates.
Ahh found it. Musta glanced over it when I skimmed through dmesg the first time.
Thank you!
[14681859.111298] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
So for the updates. How do I know when I need to install updates? I looked at releases and saw 8.2.1 was latest. Do I need to check somewhere like github?
NVM: https://koji.xcp-ng.org/builds?inherited=0&tagID=42&order=-build_id&latest=1
https://xcp-ng.org/docs/updates.html#prerequisites
FYI, system is 100% offline; so I will now subscribe to the update thread... lol
https://xcp-ng.org/forum/topic/365/updates-announcements-and-testing
Wow. Failure to read documentation. [I tried, kinda]
I am on 8.2.1; uptime 8+ months, no issues.
This is the IPMI screen output (VGA)
Recent history:
I fast copied a Win10 VM yesterday and stood it up for a user.
A tech randomed (wrong IP) into viewing the IPMI output today and pointed it out to me.
I checked:
/var/log/daemon.log
/var/log/xensource.log
/var/log/SMlog
/var/log/kern.log
dmesg
I don't see any references to a Kernel Bug in any of the logs.
Seems similar to this thread: https://xcp-ng.org/forum/topic/5174/kernel-null-pointer
@olivierlambert
Yep, thanks.
OP edited to direct link to XOA NTP documentation for those who need that.
There was no clear distinction in my mind. Apologies for any confusion caused.
So...
XOA = xen orchestra appliance
XO = xen orchestra (from sources)
I just edited my title.
Wouldn't changing NTP for both be the same? XOA just comes with a debian host OS on a virtual drive...? Who cares.
Edit: I guess maybe the XOA version comes with ntp installed?
P.S. FWIW:
The whole ecosystem is confusing and annoying to learn.
Need a single page that clearly states everything for newbies. I even forget because I don't do this constantly. Just trying to spend my morning knowledge dumping.
Set NTP in 3 places:
Steps for XO [from sources] (Xen Orchestra VM on Debian 10 Host OS):
/etc/systemd/timesyncd.confsudo timedatectl set-ntp truesudo systemctl restart systemd-timesyncd.servicesystemctl status systemd-timesyncd.servicetimedatectl timesync-statustimdatectl statusSteps for XCP-ng hypervizor host (CLI easy mode):
xsconsole
Steps for XCP-ng hardware hypervisor (CLI hard mode) :
sudo nano /etc/chrony.confserver pool.ntp.org iburstsystemctl restart chronydchronyc trackingThe other NTP posts make no sense.
ntpd -p does not work
It wants you to install the NTP package.
No thanks, my XO is offline.
Give me the default NTP service built-in.
I have already have XO installed in a Debian 10 built from sources using Tom's video guide.
It is offline. Not gonna put it online to add ntp service.
Installed: xen-tools, ssh keys, tmux, htop, git, cloned xen-xo repo, vim, update, upgrade, ssh.
Edited:
I misunderstood difference between XO and XOA. Post updated to reflect that.
XOA contains the NTP package. Mine did not. I'd say most building from sources won't either.
Somewhat useful:
https://www.freedesktop.org/software/systemd/man/timedatectl.html
https://xcp-ng.org/forum/topic/5858/setting-ntp-time-servers-from-terminal-ssh?_=1671025646276
Not useful:
https://serverfault.com/questions/913968/ntp-doesnt-seem-to-be-enabled-and-synchronised-in-debian-jessie
https://www.digitalocean.com/community/tutorials/how-to-set-up-time-synchronization-on-debian-10
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/s1-checking_the_status_of_ntp
I wanted to detail my experience installing XCP-ng on a few different systems.
Here's an older system. I'm very happy with this for what it is (I have Threadripper system that is awesome).
Interesting details:
TrueNAS13 running inside XCP-ng (Just follow Tom's Guide)
Equipment:
Chassis: CSE-825
Mobo: X9DRW-iF
Proc: 2x E5-2620
Mem: 64GB M393B2G70BH0-CK0 | DDR3 1600MHz PC3-12800R | forced to 1600
SSD: MX500, Evo870: 500GB Software Raid (XCP-NG) Boot -> UEFI:XCP-NG
HBA: Avago (Broadcom) 9311-8i
Expander: SAS825TQ (SAS2, 8xSATA)
Notes:
Live swapping SATA or SFF-8643 cables works
SilverStone Technology SFF-8643 to SATA+Sideband (CPS05-RE) does not work
0.5M is barely enough, 0.6M is just about right, 1.0M would be way better
Flashing IT over TrueNAS did not work (could not erase)
64G of RDIMM was cheaper than UDIMM due to larger stick capcaity (8x 8G vs 4x 16G)
IPMI - IP changed from GUI not from BIOS (weird)
Leaving the system powered on will make swapping/testing faster due to IPMI not needing to boot everytime you remove power (obv don't do this for system resources [cpu] [Memory, PCIe are debatable; perhaps with grounding]
XCP-NG 8.2.1 works very well https://xcp-ng.org/#easy-to-install
Updating the bios/IPMI (BMC) on the mobo is a PITA, but worth it so IPMI works better
I did IPMI through the web interface
http://blog.smalleycreative.com/linux/flashing-the-ipmi-firmware-on-a-supermicro-x9drw-if/
You have to unlock the bios for it to update
https://peterkleissner.com/2018/05/27/reverse-engineering-supermicro-ipmi/
https://www.supermicro.com/support/resources/bios_ipmi.php
http://blog.smalleycreative.com/administration/updating-the-bios-firmware-on-the-supermicro-x9drw-if/
HBA Cards:
SAS2 - 2xxx controller (Sata2, PCIE 2.0, 6Gbps)
SAS3 - 3xxx controller (SATA3, PCIE 3.0, 12Gbps)
IR = raid
IT = HBA ("initiator target")
https://forums.servethehome.com/index.php?threads/what-is-it-mode.328/#:~:text=IT mode stands for "initiator,raid formats the HBA supports.
fix boottime issues (I haven't tried this):
https://www.reddit.com/r/DataHoarder/comments/65w19b/do_hbas_increase_boot_time/
Flashing HBA (IR to IT):
-You could flash inside TrueNAS, but it didn't work for me.
-Read the sas3Flash_quickRefGuide_rev1-0 guide downloaded with the package for windows
You're looking for these downloads (PXX - replace for your version [currently 16.0, 6-26-2022]:
SAS3_UEFI_BSD_PXX
9300_8i_Package_PXX_IR_IT_FW_BIOS_for_MSDOS_Windows
you want these files:
sas3flsh.efi or sas3flash.efi
SAS9300_8i_IT.bin (you do NOT want to use the IR bin)
mptsas3.rom
mpt3x64.rom
SAS 3008 (9311-8i)
https://www.broadcom.com/support/download-search?pg=&pf=&pn=&pa=&po=&dk=9300&pl=&l=false
https://www.servethehome.com/flash-lsi-sas-3008-hba-e-g-ibm-m1215-mode/
https://www.truenas.com/community/threads/how-do-i-flash-lsi-avago-sas-9311-i8.72964/
Flashing HBA (SAS3 16.00.12.00 < version is required for TrueNAS)
File here: https://www.truenas.com/community/resources/lsi-9300-xx-firmware-update.145/
Inside TrueNAS (I don't recommend this; card gets stuck 'resetting' requiring a host reboot):
-make a pool, smb it, allow guests, open ACL
-copy the downloaded file to the pool
-ssh or shell into TN, launch mc (midnight commander)
-make folder /etc/driver
-copy /mnt/tank/pool/SAS9300_8i_IT.bin to /etc/driver
-cd /etc/driver
-sas3flash -o -f SAS9300_8i_IT.bin [resetting adapter always gets stuck]
Outside:
-copy to flashdrive you used for IR to IT [overwrite]
-UEFI boot
-sas3flash -o -f SAS9300_8i_IT.bin [I renamed my v12 to SAS9300_8i_IT_1200.bin so I could do both at the same time]
-reboot
UEFI shell:
#basically a built-in unix (?) OS that can do things for you (instead of building a bootable flashdrive or loading DOS)
-use page up/down # scroll up/down
-map # show all devices
-fs0/1/2 # will be one of the drives (usb too)
-tab # auto-completes a word
https://docstore.mik.ua/manuals/hp-ux/en/5991-1247B/ch04s13.html
https://www.intel.com/content/dam/support/us/en/documents/motherboards/server/sb/efi_instructions.pdf
https://superuser.com/questions/1412403/how-to-scroll-up-and-down-in-efi-shell
What does this post have to do with XCP-NG?
Well...TrueNAS 13.0 is running inside of it with Pass-thru HBA SAS9300-8i to 6x HDD'same
XCP-NG is running on two SSD's 500GB
NIC-teaming is setup
Hypervisor host is performing very well with a VMS and SNMP server (both are resource hogs) and a few W10/SVR19 VMs