XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login
    • Profile
    • Following 0
    • Followers 0
    • Topics 6
    • Posts 29
    • Groups 0
    C Offline
    1. Home
    2. carloum70
    3. Posts

    Posts

    Recent Best Controversial
    • RE: HA causes reboot of xcp-ng nodes

      @tjkreidl

      According to xsconsole the management interface is bond0

      Current Management Interface          
                                                    
      Device           bond0                
      MAC Address	 30:3e:a7:1d:b0:90    
      DHCP/Static IP   Static               
      IP address       172.28.4.11          
      Netmask          255.255.252.0        
      Gateway          172.28.4.1           
      Hostname         dacshyp002  
      

      According the output of xe pif-list

      uuid ( RO)                  : 0761d887-268e-5cf0-401b-a08ad7c419aa
                      device ( RO): bond0
                         MAC ( RO): 30:3e:a7:1d:b0:90
          currently-attached ( RO): true
                        VLAN ( RO): -1
                network-uuid ( RO): 28d51793-0226-cf3f-75d3-ed7c5a9d8b33
                   host-uuid ( RO): 41a2a448-a5dc-44c6-be44-c07540d75c60
      

      Let's check the network

      # xe network-list uuid=28d51793-0226-cf3f-75d3-ed7c5a9d8b33
      uuid ( RO)                : 28d51793-0226-cf3f-75d3-ed7c5a9d8b33
                name-label ( RW): mgmt-bond
          name-description ( RW): 
                    bridge ( RO): xapi1
      

      At linux level:

      # ip addr show xapi1
      15: xapi1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
          link/ether 30:3e:a7:1d:b0:90 brd ff:ff:ff:ff:ff:ff
          inet 172.28.4.11/22 brd 172.28.7.255 scope global xapi1
             valid_lft forever preferred_lft forever
      

      xapi1 is a bond of et2 and eth3. So far so good.
      But if I am running the xe-reset-networking command

      Your network will be re-configured as follows:
      
      Management interface:   eth4
      Reset interface name rules: Yes
      IP configuration mode:  dhcp
      IPv6 configuration mode:none
      
      If you want to change any of the above settings, type 'no' and re-run
      the command with appropriate arguments (use --help for a list of options).
      
      Type 'yes' to continue.
      Type 'no' to cancel.
      no
      

      So for some reason xcp-ng thinks the eth4 is management interface, which also has the ip-address 172.28.4.11/22 .
      By the way --reset-primary is not a valid option:

      # xe-reset-networking --reset-primary
      Usage: xe-reset-networking [options]
      
      xe-reset-networking: error: no such option: --reset-primary
      
      posted in Management
      C
      carloum70
    • RE: HA causes reboot of xcp-ng nodes

      I did some further investigation and I think there is something wrong with the ip-settings of the management interface.

      At the xsconsole I see the following:

      Management Network Parameters    
      
      Device        bond0         
      address       172.28.4.11    
      Netmask       255.255.252.0   
      Gateway       172.28.4.1
      

      So it's using the bond0 interface.

      # xe pif-list host-uuid=41a2a448-a5dc-44c6-be44-c07540d75c60
      
      uuid ( RO)                  : 0761d887-268e-5cf0-401b-a08ad7c419aa
                      device ( RO): bond0
                         MAC ( RO): 30:3e:a7:1d:b0:90
          currently-attached ( RO): true
                        VLAN ( RO): -1
                network-uuid ( RO): 28d51793-0226-cf3f-75d3-ed7c5a9d8b33
                   host-uuid ( RO): 41a2a448-a5dc-44c6-be44-c07540d75c60
      

      This is part of the output of xe bond-list

      # xe bond-list
      uuid ( RO)      : 79c525c3-22f9-04ce-4a80-21d18ef65ebc
          master ( RO): 0761d887-268e-5cf0-401b-a08ad7c419aa
          slaves ( RO): 7ac18886-e0a5-057b-bac0-36b909588dba ; 2f18fc9a-8c1c-dfce-9416-fa8c657c5f63 
      

      I already checked that:

      7ac18886-e0a5-057b-bac0-36b909588dba --> eth3
      2f18fc9a-8c1c-dfce-9416-fa8c657c5f63 --> eth2
      

      And now the confusing part. This is part of the output of the ip a command:

      2: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
          link/ether 30:3e:a7:1d:b0:90 brd ff:ff:ff:ff:ff:ff
      3: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
          link/ether 30:3e:a7:1d:b0:91 brd ff:ff:ff:ff:ff:ff
      4: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
          link/ether 30:3e:a7:1d:b0:92 brd ff:ff:ff:ff:ff:ff
      
      
      13: xenbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
          link/ether 30:3e:a7:1d:b0:92 brd ff:ff:ff:ff:ff:ff
          inet 172.28.4.11/22 brd 172.28.7.255 scope global xenbr4
             valid_lft forever preferred_lft forever
      

      As you can see the xenbr4 has also the management ip-address and has the same mac-address as the eth4 interface.
      OK check the following commands:

      [17:21 dacshyp002 ~]# ip addr show xapi1
      15: xapi1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
          link/ether 30:3e:a7:1d:b0:90 brd ff:ff:ff:ff:ff:ff
          inet 172.28.4.11/22 brd 172.28.7.255 scope global xapi1
             valid_lft forever preferred_lft forever
      [17:21 dacshyp002 ~]# ip addr show xenbr4
      13: xenbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
          link/ether 30:3e:a7:1d:b0:92 brd ff:ff:ff:ff:ff:ff
          inet 172.28.4.11/22 brd 172.28.7.255 scope global xenbr4
             valid_lft forever preferred_lft forever
      

      And If I look at the routing table:

      # ip route
      default via 172.28.4.1 dev xapi1 
      172.18.8.0/23 dev xenbr0 proto kernel scope link src 172.18.8.11 
      172.18.10.0/23 dev xenbr7 proto kernel scope link src 172.18.10.11 
      172.28.4.0/22 dev xenbr4 proto kernel scope link src 172.28.4.11 
      172.28.4.0/22 dev xapi1 proto kernel scope link src 172.28.4.11 
      

      According to me the node has 2 interfaces xenbr4 / xapi1 with the same ip-address and also 2 different routes for the same ip-range 172.28.4.0/22.
      And that's why the heartbeat has some issues. Can someone confirm this ?
      By the way I really don't know how this happened.

      posted in Management
      C
      carloum70
    • RE: HA causes reboot of xcp-ng nodes

      I also checked the xha.log on the dacshyp002:

      Sep 21 11:39:49 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=30408.
      Sep 21 11:40:09 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 11:40:29 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=31466.
      Sep 21 11:40:50 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 11:46:30 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=35178.
      Sep 21 11:46:50 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 11:47:10 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=33233.
      Sep 21 11:47:30 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 11:55:51 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=40853.
      Sep 21 11:56:11 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 11:56:31 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=32914.
      Sep 21 11:56:51 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 11:59:31 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=33377.
      Sep 21 11:59:51 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:01:51 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=38588.
      Sep 21 12:02:12 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:10:12 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=38125.
      Sep 21 12:10:12 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=38323.
      Sep 21 12:10:32 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:19:33 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=43934.
      Sep 21 12:19:53 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:20:13 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=32989.
      Sep 21 12:20:33 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:21:13 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=42077.
      Sep 21 12:21:33 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:21:53 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=31325.
      Sep 21 12:22:13 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:25:54 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=34497.
      Sep 21 12:26:14 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:26:34 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=32558.
      Sep 21 12:26:54 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:27:14 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=30616.
      Sep 21 12:27:34 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:31:14 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=33975.
      Sep 21 12:31:34 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:38:15 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=34797.
      Sep 21 12:38:35 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:53:56 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[1].time_since_last_hb=37000.
      Sep 21 12:53:56 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=40176.
      Sep 21 12:54:16 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:56:16 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=36380.
      Sep 21 12:56:36 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      Sep 21 12:56:57 CEST 2026 [warn] SC: (script_service_do_query_liveset) reporting "heartbeat approaching timeout". host[2].time_since_last_hb=31439.
      Sep 21 12:57:17 CEST 2026 [info] SC: (script_service_do_query_liveset) "Heartbeat approaching timeout" turned FALSE 
      

      I can conclude the heartbeat communication is generally unstable in my HA pool. Tomorrow I check the logging of our switch/router.

      posted in Management
      C
      carloum70
    • RE: HA causes reboot of xcp-ng nodes

      dacshyp003 booted at

      [Mon Sep 21 05:44:37 2026] Linux version 4.19.0+1 (mockbuild@1efe98ffea2245bcb4c11893890153ac) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)) #1 SMP Thu Aug 13 13:06:11 UTC 2026
      

      This is the part of the xensource logging of the dacshyp001 before the reboot:

      Sep 21 11:17:52 dacshyp001 xapi: [debug||17 ha_monitor|HA monitor D:58d1715e0726|xapi_ha] The node we think is the master is still alive and marked as master; this is OK
      ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
      

      Same behavior on dacshyp003.
      I also attached the xha.log from both nodes.

      xha_log_001.txt xha_log_003.txt

      posted in Management
      C
      carloum70
    • HA causes reboot of xcp-ng nodes

      Hi all,

      I need some help regarding HA.
      I have a 3 node cluster running xcp-ng 8.3 (002 is the master)

      [13:55 dacshyp002 ~]# xe host-list
      uuid ( RO)                : 6b99f1ab-6f4c-4a8d-b766-d16a8a942bdf
                name-label ( RW): dacshyp003
          name-description ( RW): Default install
      
      
      uuid ( RO)                : d99e150c-079a-4092-8909-ad1a36e07dec
                name-label ( RW): dacshyp001
          name-description ( RW): Default install
      
      
      uuid ( RO)                : 41a2a448-a5dc-44c6-be44-c07540d75c60
                name-label ( RW): dacshyp002
          name-description ( RW): Default install
      

      Last Thursday I enabled HA on the pool and also on some om the VM's. On Friday I also did an "rolling pool update"
      Today I had a "spontaneous" reboot of the dacshyp001 and dacshyp003.
      Boot time of dacshyp001:

      [Mon Sep 21 11:21:31 2026] Linux version 4.19.0+1 (mockbuild@1efe98ffea2245bcb4c11893890153ac) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)) #1 SMP Thu Aug 13 13:06:11 UTC 2026
      

      Around that time I also see the following message in the xensource.log:

      Sep 21 11:21:08 dacshyp002 xapi: [debug||20398 ha_monitor|HA monitor D:7b5068fa6bfc|xapi_ha_vm_failover] Setting host dacshyp001 to dead
      Sep 21 11:21:28 dacshyp002 xapi: [debug||20398 ha_monitor|HA monitor D:7b5068fa6bfc|xapi_ha_vm_failover] Setting host dacshyp001 to dead
      Sep 21 11:21:48 dacshyp002 xapi: [debug||20398 ha_monitor|HA monitor D:7b5068fa6bfc|xapi_ha_vm_failover] Setting host dacshyp001 to dead
      

      I will attached all logging around this time.
      I am trying to understand what triggered the reboot and how I can troubleshoot this.

      I am running XO community commit 6a441 . I am aware this is not the latest version. Before upgrading I want to know the cause of the reboot.

      Also ignore the error:

      /var/lib/xcp/xapi|dispatch:VDI.get_by_uuid D:1f5b4c8d329e|backtrace] VDI.get_by_uuid D:9df5c0255224 failed with exception Db_exn.Read_missing_uuid("VDI", "", "9219113f-65e4-4368-b42d-1b8bdc7614a9")
      

      I will create another ticket for this.
      HA-problem.txt
      Thanks in advance.
      Carlo

      posted in Management
      C
      carloum70
    • RE: Error while scanning disk

      @poddingue https://github.com/vatesfr/xen-orchestra/issues/9779

      posted in Backup
      C
      carloum70
    • RE: Error while scanning disk

      Is there some update? Can this issue be reproduced?

      posted in Backup
      C
      carloum70
    • RE: Error while scanning disk

      @ataxyanetwork I managed to install XOA (Current version: 6.2.2 - XOA build: 20251219) and did some more testing.

      I am experiencing the same issue when trying to restore from an existing backup.

      When I create a new backup job, I can perform a filelevel restore without any problems. However, when I rerun the job, I am no longer able to restore from the most recent backup. It only works with the initial backup.

      Can you also reproduce this ?

      posted in Backup
      C
      carloum70
    • RE: Error while scanning disk

      @AtaxyaNetwork This is not going to work because our nodes don't have access to the internet.
      If it works using XOA, what conclusion can we draw? Migrate to XOA ?

      posted in Backup
      C
      carloum70
    • RE: Error while scanning disk

      @AtaxyaNetwork Thank you for the quick response.
      "Are you running an XO source with LVM for the OS ?" --> no

      You did the restore using XOA instead of XO from source ?
      To Install XOA I have to install a new instance and import the config of my current XO ?
      I will test this tomorrow.

      posted in Backup
      C
      carloum70
    • RE: Error while scanning disk

      @AtaxyaNetwork Sorry forgot to add this to my topic.

      XOA - commit 3ed8c
      VM's are based on Debain13
      UEFI
      Backup mode: Delta backup

      ~# lsblk -f 
      NAME              FSTYPE      FSVER    LABEL UUID                                   FSAVAIL FSUSE% MOUNTPOINTS
      xvda                                                                                               
      ├─xvda1           vfat        FAT32          0E7D-17B9                               965.3M     1% /boot/efi
      ├─xvda2           ext4        1.0            bf2a2e01-b9ff-4b93-a824-ba7562461c15      1.3G    20% /boot
      ├─xvda3           LVM2_member LVM2 001       v2o7yJ-51Hc-sO1m-ubrj-YCvP-7Zo2-zOjxzZ                
      │ ├─vgdata-lvroot ext4        1.0            d0e71b5a-5404-4b18-9249-eb222a6d346d     22.6G     6% /
      │ ├─vgdata-lvhome ext4        1.0            5aa25f3d-708d-4708-8405-a97391e2e989      4.3G     0% /home
      │ └─vgdata-lvvar  ext4        1.0            e82f5584-810b-40e2-a6e6-ed05805146db      3.5G    30% /var
      └─xvda4           swap        1              6124f717-58cf-47a8-b268-4c99dcd495d0                  [SWAP]
      
      posted in Backup
      C
      carloum70
    • Error while scanning disk

      Hi all,

      First of all, I am aware that there are still issues with file-level restores and LVM. However, for some unknown reason, I am now encountering a new error when selecting the disk during the restore procedure.- Error while scanning disk

      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]: 2026-04-16T09:08:34.366Z xo:api WARN admin | backupNg.listPartitions(...) [214ms] =!> Error: Command failed: vgchange -an vgdata
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]: File descriptor 23 (/var/lib/xo-server/data/leveldb/LOG) leaked on vgchange invocation. Parent PID 3124716: node
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]: File descriptor 24 (/var/lib/xo-server/data/leveldb/LOCK) leaked on vgchange invocation. Parent PID 3124716: node
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]: File descriptor 25 (/var/lib/xo-server/data/leveldb/000379.log) leaked on vgchange invocation. Parent PID 3124716: node
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]: File descriptor 28 (/var/lib/xo-server/data/leveldb/MANIFEST-000377) leaked on vgchange invocation. Parent PID 3124716: node
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]: File descriptor 37 (/dev/fuse) leaked on vgchange invocation. Parent PID 3124716: node
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]:   WARNING: Not using device /dev/loop0 for PV v2o7yJ-51Hc-sO1m-ubrj-YCvP-7Zo2-zOjxzZ.
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]:   WARNING: PV v2o7yJ-51Hc-sO1m-ubrj-YCvP-7Zo2-zOjxzZ prefers device /dev/xvda3 because device is used by LV.
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]:   Logical volume vgdata/lvroot contains a filesystem in use.
      Apr 16 11:08:34 dacsvm-xoa xo-server[3124716]:   Can't deactivate volume group "vgdata" with 3 open logical volume(s)
      

      Is this also a known issue ?
      Do you have an estimate for when LVM support will be available? This is currently a deal breaker for us, as our infrastructure (running on XCP-ng) is becoming increasingly business-critical.
      If this is not supported in the near future, we may need to consider alternative backup solutions.

      XOA - commit 3ed8c
      VM's are based on Debain13
      Backup mode: Delta backup

      Regards,
      Carlo

      posted in Backup
      C
      carloum70
    • RE: OIDC login - Internal Server Error

      Thanks for the reply's! This really helps.

      I added the

      console.log('OIDC profile:', JSON.stringify(profile, null, 2))
      

      and then tried with the following:

      username field: email
      scope: email

      logging:

      mrt 30 12:34:57 vm-xoa xo-server[2747568]: OIDC profile: {
      mrt 30 12:34:57 vm-xoa xo-server[2747568]:   "id": "38882f04f015223135313da0b919cb3d67bf4fbc@sram.surf.nl"
      mrt 30 12:34:57 vm-xoa xo-server[2747568]: }
      mrt 30 12:34:57 vm-xoa xo-server[2747568]: Cannot read properties of undefined (reading '0')
      

      username field: uid
      scope: uid

      logging:

      mrt 30 12:35:54 vm-xoa xo-server[2747568]: OIDC profile: {
      mrt 30 12:35:54 vm-xoa xo-server[2747568]:   "id": "38882f04f015223135313da0b919cb3d67bf4fbc@sram.surf.nl"
      mrt 30 12:35:54 vm-xoa xo-server[2747568]: }
      mrt 30 12:35:54 vm-xoa xo-server[2747568]: Expected values to be strictly equal:
      mrt 30 12:35:54 vm-xoa xo-server[2747568]: + actual - expected
      mrt 30 12:35:54 vm-xoa xo-server[2747568]: + 'undefined'
      mrt 30 12:35:54 vm-xoa xo-server[2747568]: - 'string'
      

      So it seems that in both cases we only receive the "sub" from the scope openid from surf. Which is here named "id". Is this translated by xo?

      Then I applied the patch from @olivierlambert . Therafter we were able to login by using "id" as Username field, "sub" returned an error. The user 38882f04f015223135313da0b919cb3d67bf4fbc@sram.surf.nl was then created.

      If I use email or uid we get now better logging:

      mrt 30 12:41:14 vm-xoa xo-server[2747760]: Could not find username: field "uid" is missing from the OIDC profile. Ensure the required scopes are configured and granted by your identity provider.
      mrt 30 12:42:27 vm-xoa xo-server[2747760]: Could not find username: field "email" is missing from the OIDC profile. Ensure the required scopes are configured and granted by your identity provider.
      

      We will check if we have to enable/allow additional claims from the authentication provider to be available and let you now.

      posted in Advanced features
      C
      carloum70
    • RE: OIDC login - Internal Server Error

      @HeMaN

      We would like to know that too.

      It seems that xo is receiving something, but not what it is expecting. But unfortunately we cannot see what it receives.

      With a saml tracer plugin I can see data that my browser exchanges with Surf and there are the field that are used/needed, like the uid and the mail.

      browser-reply.png

      But from my understanding xo requests its own data from surf based on token from the web session.

      posted in Advanced features
      C
      carloum70
    • RE: OIDC login - Internal Server Error

      We are running Xen Orchestra with commit c3dcb and the auth-oidc (v0.4.2) plugin.

      The users that login are unique and not yet present as local users.

      The OICD provider is SURF with SRAM: https://www.surf.nl/en/services/identity-access-management/surf-research-access-management

      They support the following attributes/scopes: https://servicedesk.surf.nl/wiki/spaces/IAM/pages/74226142/Attributes+in+SRAM

      There are some IPs that need to be accessable: https://servicedesk.surf.nl/wiki/spaces/IAM/pages/74226067/IP+addresses#IPaddresses-OIDC . Outgoing traffic from the server to port 443 is open and works.

      We did try several settings for the Username field and scopes.

      For example the following:

      plugin.png

      This should create a user R123456789

      The logging shows the following:

      mrt 27 09:29:48 vm-xoa xo-server[2641104]: Expected values to be strictly equal:
      mrt 27 09:29:48 vm-xoa xo-server[2641104]: + actual - expected
      mrt 27 09:29:48 vm-xoa xo-server[2641104]: + 'undefined'
      mrt 27 09:29:48 vm-xoa xo-server[2641104]: - 'string'
      

      If we change it to:

      plugin-2.png

      The following shows up:

      mrt 27 09:32:21 vm-xoa xo-server[2641104]: Cannot read properties of undefined (reading '0')
      

      I hope this helps to understand the problem. Thanks.

      posted in Advanced features
      C
      carloum70
    • OIDC login - Internal Server Error

      We are trying to use the OIDC auth plugin to enable login to our Xen Orchestra without local accounts.

      We registered a client with our identity provider and got a client id, client secret and the auto-discovery url. That we used to configure the plugin.

      However, if we login we get redirected back from the identity provider to the XO callback url and receive then an "Internal Server Error"

      The callback URL is as follow:

      https://xoa.domain.com/signin/oidc/callback?state=STRING&scope=profile+openid&code=STRING&iss=https://identity-provider.url&client_id=XXXXX

      In the log file we see then the following 4 lines:

      mrt 25 12:29:25 vm-xoa xo-server[2618522]: Expected values to be strictly equal:
      mrt 25 12:29:25 vm-xoa xo-server[2618522]: + actual - expected
      mrt 25 12:29:25 vm-xoa xo-server[2618522]: + 'undefined'
      mrt 25 12:29:25 vm-xoa xo-server[2618522]: - 'string'
      

      If we change both the username field and the scope to email, we get the same Internal Server Error, but with a different single log line:

      mrt 25 13:18:04 vm-xoa xo-server[2618522]: Cannot read properties of undefined (reading '0')
      

      Because we are getting redirected back from our identity provider to Xen Orchestra we guess that the issue is not there. We also get in the browser a SAML response with the userdata.

      Running a wireshark on the server shows also traffic between Xen Orchestra and the identity provider, but unfortunately we cannot look in the contents of that traffic stream.

      Setting the log level to debug does unfortunately not produce more (error) output.

      We are running Xen Orchestra with commit c3dcb and the auth-oidc (v0.4.2) plugin

      Is there an other way to figure out what is going wrong?

      posted in Advanced features
      C
      carloum70
    • RE: Unable to configure Network IP inside the VM throgh API

      What is the output of the following command:

      cloud-init schema --system
      
      posted in Infrastructure as Code
      C
      carloum70
    • RE: Unable to configure Network IP inside the VM throgh API

      I am using terraform to setup a VM but it's the same approach for cloud-init:
      cloud_config

      #cloud-config
      preserve_hostname: false
      hostname: ${hostname} 
      create_hostname_file: true
      users:
        - name: ansible
          groups:
            - sudo
          sudo:
            - ALL=(ALL) NOPASSWD:ALL
          shell: /bin/bash
          ssh_authorized_keys:
            - "ssh-ed25519 somekeyl"
            - "ssh-ed25519 otherkey"
      

      network_config:

      #cloud-config
      network:
        version: 1
        config:
          - type: physical
            name: enX0
            subnets:
              - type: static
                address: ${ip_address}
                netmask: ${netmask}
                gateway: ${gateway}
                dns_nameservers:
      %{ for dns_server in dns ~}
                  - ${dns_server}
      %{ endfor ~}
      

      Maybe a silly question: Your template supports cloud-init ?

      posted in Infrastructure as Code
      C
      carloum70
    • RE: Unable to configure Network IP inside the VM throgh API

      @amititre331
      If you want to configure ip settings using cloud-init you have to use the option network_config instead of cloud_config.

      Unbrand_CreateVmBody_{
      memory	
      name_description	[...]
      name_label*	[...]
      clone	[...]
      gpuGroup	[...]
      vgpuType	[...]
      autoPoweron	[...]
      vifs	[...]
      copyHostBiosStrings	[...]
      template*	[...]
      affinity	[...]
      vdis	[...]
      install	{...}
      cloud_config	[...]
      network_config	string
      boot	[...]
      destroy_cloud_config_vdi	[...]
      }
      

      By the way, do not share your tokens on the internet 😉

      posted in Infrastructure as Code
      C
      carloum70
    • RE: Terraform and disk migrations

      This approach isn’t entirely foolproof since I can’t use a wildcard, and I don’t know how many disks will be attached to the VM. For now, it will never exceed two disks, so I can explicitly include both in the ignore_changes statement. That’s an acceptable solution for us. Thanks for your support.

      posted in Infrastructure as Code
      C
      carloum70