Existing AD Users Cannot Login to XOCE but New Users Can

kagbasi-wgsdac

Good-day Folks,

I'm having an issue with LDAP login against an Active Directory domain controller. Fortunately, I am the only admin, so it hasn't been a major problem (since I can still login with a local account). Anybody else out there running into this problem?

MY ENVIRONMENT:
xo-server 5.113.0 / Xen Orchestra, commit c0465 / xo-web 5.116.0

I've read through the following posts and confirmed that my settings are correct and should be working:

Here's the output of the test-cli.js script for my existing account and a test account I created (after the problem started

FOR MY USER ACCOUNT (kagbasi)

root@mydomain-SV-XO1:/opt/xo/xo-server/node_modules/xo-server-auth-ldap/dist# ./test-cli.js ldap.cache.conf
? URI ldap://dc01.mydomain.net
? fill optional Certificate Authorities? No
? fill optional Check certificate? No
? fill optional Use StartTLS? No
? Base OU=MyOU,DC=mydomain,DC=net
? fill optional Credentials? Yes
? Credentials > dn xxXOC@mydomain.net
? Credentials > password ***
? fill optional User filter? Yes
? User filter (&(sAMAccountName={{name}})(memberOf=CN=IT_Linux_Admins,OU=Groups,OU=MyOU,DC=mydomain,DC=net))
? fill optional ID attribute? Yes
? ID attribute sAMAccountName
? fill optional Synchronize groups? No
configuration saved in ./ldap.cache.conf
? Username kagbasi
? Password [hidden]
2023-05-14T08:33:48.354Z xo:xo-server-auth-ldap DEBUG attempting to bind with as xxXOC@mydomain.net...
2023-05-14T08:33:48.369Z xo:xo-server-auth-ldap DEBUG successfully bound as xxXOC@mydomain.net
2023-05-14T08:33:48.369Z xo:xo-server-auth-ldap DEBUG searching for entries...
2023-05-14T08:33:48.375Z xo:xo-server-auth-ldap DEBUG 1 entries found
2023-05-14T08:33:48.375Z xo:xo-server-auth-ldap DEBUG attempting to bind as CN=Agbasi\, Kismet,OU=General Users,OU=Users,OU=MyOU,DC=mydomain,DC=net
2023-05-14T08:33:48.378Z xo:xo-server-auth-ldap DEBUG failed to bind as CN=Agbasi\, Kismet,OU=General Users,OU=Users,OU=MyOU,DC=mydomain,DC=net: 80090308: LdapErr: DSID-0C090434, comment: AcceptSecurityContext error, data 569, v4f7c Code: 0x31
2023-05-14T08:33:48.378Z xo:xo-server-auth-ldap DEBUG could not authenticate kagbasi
root@mydomain-SV-XO1:/opt/xo/xo-server/node_modules/xo-server-auth-ldap/dist#

FOR A TEST USER ACCOUNT (test123)

root@mydomain-SV-XO1:/opt/xo/xo-server/node_modules/xo-server-auth-ldap/dist# ./test-cli.js ldap.cache.conf
? URI ldap://dc01.mydomain.net
? fill optional Certificate Authorities? No
? fill optional Check certificate? No
? fill optional Use StartTLS? No
? Base OU=MyOU,DC=mydomain,DC=net
? fill optional Credentials? Yes
? Credentials > dn xxXOC@mydomain.net
? Credentials > password ***
? fill optional User filter? Yes
? User filter (&(sAMAccountName={{name}})(memberOf=CN=IT_Linux_Admins,OU=Groups,OU=MyOU,DC=mydomain,DC=net))
? fill optional ID attribute? Yes
? ID attribute sAMAccountName
? fill optional Synchronize groups? No
configuration saved in ./ldap.cache.conf
? Username test123
? Password [hidden]
2023-05-14T08:43:05.780Z xo:xo-server-auth-ldap DEBUG attempting to bind with as xxXOC@mydomain.net...
2023-05-14T08:43:05.795Z xo:xo-server-auth-ldap DEBUG successfully bound as xxXOC@mydomain.net
2023-05-14T08:43:05.795Z xo:xo-server-auth-ldap DEBUG searching for entries...
2023-05-14T08:43:05.801Z xo:xo-server-auth-ldap DEBUG 1 entries found
2023-05-14T08:43:05.801Z xo:xo-server-auth-ldap DEBUG attempting to bind as CN=Test User,OU=General Users,OU=Users,OU=MyOU,DC=mydomain,DC=net
2023-05-14T08:43:05.803Z xo:xo-server-auth-ldap INFO successfully bound as CN=Test User,OU=General Users,OU=Users,OU=MyOU,DC=mydomain,DC=net => test123 authenticated
2023-05-14T08:43:05.803Z xo:xo-server-auth-ldap DEBUG {
  "dn": "CN=Test User,OU=General Users,OU=Users,OU=MyOU,DC=mydomain,DC=net",
  "objectClass": [
    "top",
    "person",
    "organizationalPerson",
    "user"
  ],
  "cn": "Test User",
  "sn": "User",
  "givenName": "Test",
  "distinguishedName": "CN=Test User,OU=General Users,OU=Users,OU=MyOU,DC=mydomain,DC=net",
  "instanceType": "4",
  "whenCreated": "20230514083604.0Z",
  "whenChanged": "20230514083657.0Z",
  "displayName": "Test User",
  "uSNCreated": "696212",
  "memberOf": "CN=IT_Linux_Admins,OU=Groups,OU=MyOU,DC=mydomain,DC=net",
  "uSNChanged": "696231",
  "name": "Test User",
  "objectGUID": "~\b\u000e��\u001f�E����\r�,�",
  "userAccountControl": "512",
  "badPwdCount": "0",
  "codePage": "0",
  "countryCode": "0",
  "badPasswordTime": "0",
  "lastLogoff": "0",
  "lastLogon": "0",
  "pwdLastSet": "133285269646375752",
  "primaryGroupID": "513",
  "objectSid": "\u0001\u0005\u0000\u0000\u0000\u0000\u0000\u0005\u0015\u0000\u0000\u0000�A�\u0015�d�G�:��`\u0006\u0000\u0000",
  "accountExpires": "9223372036854775807",
  "logonCount": "0",
  "sAMAccountName": "test123",
  "sAMAccountType": "805306368",
  "userPrincipalName": "test123@mydomain.net",
  "objectCategory": "CN=Person,CN=Schema,CN=Configuration,DC=mydomain,DC=net",
  "dSCorePropagationData": "16010101000000.0Z",
  "lastLogonTimestamp": "133285270174344684"
}
root@mydomain-SV-XO1:/opt/xo/xo-server/node_modules/xo-server-auth-ldap/dist#

olivierlambert

Hi,

As stated in the doc, always start by using the latest commit on master: https://xen-orchestra.com/docs/community.html#report-a-bug

kagbasi-wgsdac

@olivierlambert My apologies. I've updated and retested, the issue persists.

MY ENVIRONMENT:
xo-server 5.114.1 / Xen Orchestra, commit 01ba1 / xo-web 5.117.1

I think I may have found the problem, if someone can confirm that would be awesome. Turns out if the number of groups the user belongs to exceeds seven (7), LDAP authentication fails. I arrived at this conclusion by creating a new user (by copying the failing user) and removing the groups, one by one (testing in between each removal) until I was successful.

Unfortunately, I was not able to successfully reproduce the problem in the reverse (i.e., by adding the same user to eight (8) security groups). I even added it to more groups, and authentication still succeeds. Which seems to suggest that something is cached on the Linux side. I performed some additional tests as follows:

TEST #1:

Created new user and added them to seven (7) security groups - LDAP Auth Successful
Added an eight security group to the same user and retested - LDAP Auth Successful

TEST #2:

Created a new user and added them to eight (8) security groups - LDAP Auth Successful

TEST #3:

Created a new user and added them to nine (9) security groups - LDAP Auth Successful

I repeated the above tests until I got to TEST #9 (where the user was added to fifteen security groups), and still couldn't reproduce the problem.

I attempted to read through the plugin's code on GitHub, but couldn't make sense of it (due to my limited understanding of JavaScript).

So, a couple of questions:

Is there a limit to how many security groups a user can belong to?
Is the initial successful query being cached and re-used, somehow? If so, where can I find and clear it?

olivierlambert

On "Linux side", it's not Linux but a NodeJS library that handles the LDAP protocol. @julien-f do you think it worth opening a bug report on the passport Github project?

kagbasi-wgsdac

@olivierlambert Aah, good to know. That explains why I couldn't find any instances of winbind or SSSD...lol.

Anyway, I'm willing to help in any capacity to help further this along. So please let me know if you need me to pull any logs or screenshots, and I'll gladly do it.

kagbasi-wgsdac

@olivierlambert This issue still persists. I've updated my XOCE instance; I am currently at the following versions:

xo-server: v5.118.0
xo-web: v5.121.0
Commit: 996ab
auth-ldap: v0.10.7

Should I file a bug report? (I found a similar issue: https://github.com/vatesfr/xen-orchestra/issues/3846)

olivierlambert

Did you read the issue you posted? Maybe you have the same problem

kagbasi-wgsdac

@olivierlambert LOL, of course I did.

I considered the possibility of account lockouts but I've carefully checked each time the test failed and neither the bind account nor the test account were locked.

olivierlambert

So next step is to try with XOA. It's weird because we don't have other reports, and usually with LDAP, the context is almost everything (configuration, accounts, groups…)

kagbasi-wgsdac

@olivierlambert

As instructed, I deployed XOA, grabbed a trial license, and updated it to the latest version then tried again; same results. Getting the following error for my admin account (same one I use to log into every other workstation on the network

Code: -32000
Message: could not authenticate user
{
  "message": "could not authenticate user",
  "name": "Error",
  "stack": "Error: could not authenticate user\n    at /usr/local/lib/node_modules/xo-server-auth-ldap/src/index.js:254:15\n    at default.testPlugin (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/plugins.mjs:280:5)\n    at Xo.test (file:///usr/local/lib/node_modules/xo-server/src/api/plugin.mjs:109:3)\n    at Api.#callApiMethod (file:///usr/local/lib/node_modules/xo-server/src/xo-mixins/api.mjs:417:20)"
}

If I then use one of the test accounts I had created AFTER I started noticing the failed attempts (i.e., after 05/15/2023), it works.

Plugin test
The test appears to be working.

Prior to testing, I logged into the Domain Controller and verified that the service account I'm using to bind to AD was not locked. I'm willing to send whatever screenshots or outputs you need if it helps to get to the bottom of this.

olivierlambert

@julien-f can you remind me the name of the CLI tool to debug wrong LDAP config?

julien-f

@olivierlambert xo-server-auth-ldap.

kagbasi-wgsdac

@olivierlambert

I had already posted the output of the test-cli.js utility at the beginning of this thread, however, if you want I can do it again just to re-confirm things. Just let me know, thanks.

olivierlambert

Okay so to me it's either a weird configuration thing or a library problem in your context.

I don't know what else to do form a community point of view. You might ask on Passport library if the problem rings a bell.

@julien-f is there anything else you think we can do as far it is community support?

kagbasi-wgsdac

@olivierlambert I am very grateful for the assistance thus far. I definitely understand that your time is money, and I'm willing to approach my Church Leadership with a request to purchase the Enterprise License. However, do you offer any discounts off the annual subscription for Non-Profits Organizations? I'd be happy to take this discussion offline if you prefer (kagbasi at wgsdac.org).

Secondly, is this the correct GitHub project for the passport library you mentioned: https://github.com/vesse/passport-ldapauth/issues ? If so, I'll try posting a question there as well, as you've suggested.

olivierlambert

Checking with @julien-f

julien-f

@kagbasi-wgsdac From what I understand, there is no issue in the XO plugin itself (because it's working in many cases), it appears to be related to your LDAP/AD configuration, and unfortunately we cannot help regarding this as each LDAP/AD can be configured differently

The library we are using to connect to LDAP is https://github.com/ldapts/ldapts but I don't think that will help because it's more likely something on your server's side.

kagbasi-wgsdac

@julien-f Hmm, that's a bit disappointing that there isn't much you guys can do to help. No worries, I'll keep testing. My AD environment works fine, since the accounts in question are not locked and are being used daily. The only place they're failing is in XOCE or XOA.

It's gotta be something with how the library is handling characters in either the username or the password. I'll keep testing until I can find a repeatable pattern.

Thanks to both you and @olivierlambert for the help thus far.

kagbasi-wgsdac

RESOLVED — root cause found, three years later. Leaving a full write-up for anyone who lands here from a search.

Short version: this was never an XO bug, and it was never intermittent. The answer was sitting in the very first test-cli.js output I posted back in May 2023, and I misread it — as did everyone else in this thread, myself very much included.

The line that mattered

failed to bind as CN=Agbasi\, Kismet,...:
80090308: LdapErr: DSID-0C090434, comment: AcceptSecurityContext error, data 569, v4f7c

We all pattern-matched AcceptSecurityContext error to "bad credentials" and moved on. But the meaning is entirely carried by the data field, which is the underlying Win32 status in hex:

data 52e = 0x52E = 1326 = ERROR_LOGON_FAILURE — this is the "wrong password" one
data 525 = 1317 = ERROR_NO_SUCH_USER
data 532 = 1330 = password expired
data 775 = 1909 = account locked out
data 569 = 0x569 = 1385 = ERROR_LOGON_TYPE_NOT_GRANTED

I was getting 569, not 52e. My password was correct all along. AD validated it, then refused the logon type.

Why that happens

xo-server-auth-ldap verifies a password the only way LDAP allows — it re-binds to the directory as the user. Against Active Directory, an LDAP simple bind to a DC is processed as a Type 3 (network) logon on that DC.

So if an account is caught by "Deny access to this computer from the network" (SeDenyNetworkLogonRight) in the Default Domain Controllers Policy (or any other WINNING GPO, for that matter), it cannot complete an LDAP bind — no matter how correct the password is, and no matter which LDAP client is asking.

My environment uses a tiered admin model. Non-domain-admin admin groups are explicitly denied network logon to the DCs. My admin account is in those groups. Hence 569, every single time, by design.

Why it looked intermittent

It wasn't. I sampled it either side of a config change.

I could prove a bind had succeeded recently, because my LDAP-only XO account (no local password on the record at all) minted an API token on 28 July. Then on 31 July I restored RBAC settings on the Default Domain Controllers Policy that had drifted at some
point — I found that during unrelated PKI work. GptTmpl.inf last-write confirms it. The token's last successful use is about eleven hours before that edit.

Two deterministic states, one config change in the middle. That's the whole "intermittency."

My 2023 "seven security groups" theory was wrong

For the record, since it's still up there and someone will find it: I removed group memberships one at a time until auth worked, and concluded there was a membership count limit. There isn't. My own control test disproved it at the time — adding fifteen groups
never reproduced the failure — and I should have taken that seriously instead of filing it under "weird." The variable was never the count. It was which group. One of the removals happened to drop the account out of a denied group.

My other closing theory in this thread — special-character handling in the username or password — was also wrong. Getting 569 back proves AD parsed the escaped DN (CN=Agbasi\, Kismet), found the object, and got as far as evaluating the password. A mangled DN gives you 525 or a DN syntax error, not a logon-rights rejection. ldapts and passport were behaving correctly throughout.

How to check this in 60 seconds

Run the plugin test CLI and note the data value. Convert hex → decimal, look it up in Microsoft's System Error Codes list.
On the DC, look for Security event 4625 with Sub Status 0xC000015B (STATUS_LOGON_TYPE_NOT_GRANTED).
Fastest test of all — from a workstation, as the affected account: net use \\dc01\sysvol. If network logon to the DC is denied, this fails too, and you've confirmed it without touching XO at all.

Check the policy directly:

$p = "\\mydomain.net\SYSVOL\mydomain.net\Policies\{6AC1786C-016F-11D2-945F-00C04fB984F9}" + "\Machine\Microsoft\Windows NT\SecEdit\GptTmpl.inf"
Select-String -Path $p -Pattern "SeDenyNetworkLogonRight|SeNetworkLogonRight"

Resolve the SIDs and see whether your user is in any of the denied groups.

Also worth checking your grant side: if Access this computer from the network doesn't list Authenticated Users directly, ordinary users are probably getting it transitively via Pre-Windows 2000 Compatible Access. Worth confirming before you assume a plain
non-privileged account will work.

What I am NOT doing

Removing those groups from the deny right. It's doing exactly what I rebuilt it to do. Restoring an app login by handing admin groups network access to the DCs for SMB/RPC/LDAP is a bad trade, and I'd just be undoing my own remediation.

Fix

Interim: local XO accounts for the admins who need them. No AD objects created, nothing to unwind later, per-user attribution preserved in the audit log.
Long term: federate XO through Keycloak (OIDC) instead of LDAP. Kerberos ticket issuance is a KDC service operation and is not gated by SeNetworkLogonRight — which is exactly why these accounts log into workstations all day while failing an LDAP bind.

️ Important if you go the Keycloak route: Keycloak's LDAP user federation validates passwords by doing an LDAP bind. Configure it that way and you'll hit data 569 inside Keycloak instead of inside XO and gain nothing. Password validation has to be delegated to
Kerberos/GSSAPI.

This will bite you on anything else you point at LDAP too — Bitwarden, NPM, TrueNAS, the lot. Worth solving once at the IdP.

One request for Vates

@olivierlambert @julien-f — you were right that it was environmental, and I owe you both thanks for the time you put in back in 2023.

That said, there's a real (small) improvement available here. xo-server collapses every auth provider exception into a generic invalid credentials, and the plugin only emits the actual AD error at DEBUG. The DC told us precisely what was wrong on the very first
attempt — it just never reached anywhere a user would look.

Surfacing the LDAP result code and the AD data sub-code at INFO on failure, and in the plugin test output in the UI, would turn this class of problem from a multi-year hunt into a single-session diagnosis. Happy to open an issue on GitHub with the full reproduction if that's useful.

Hope this saves someone else three years. If you found this thread by searching data 569, *ERROR_LOGON_TYPE_NOT_GRANTED**, or "LDAP invalid credentials but password is correct" — check your Deny access to this computer from the network user right first. That's almost certainly it.