NullPointerException During LDAP User Authentication in Dremio with "oauth+ldap" coordinator.web.auth.type

lewis133 · November 25, 2024, 10:44pm

Hi all,

I am encountering a NullPointerException when attempting to authenticate users via LDAP in Dremio. Although the LDAP integration appears to retrieve user groups correctly, the authentication process fails immediately after.

Log
The user lookup appears to retrieve the user’s groups successfully as I see them printed to the log, for example:

2024-11-24 21:02:03,164 [qtp1830261681-252] DEBUG c.d.extusr.ldap.LdapUserProvider - Looking up user using filter [(&(cn=Jordan Lewis)(&(objectClass=user)(cn=Jordan Lewis)))] with base [DC=###,DC=####,DC=###] and scope [SUB] took 38 ms
2024-11-24 21:02:03,168 [qtp1830261681-252] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=ag_kasm_usr,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=###] took 2 ms

However, the process fails with a NullPointerException when attempting to load the user:

2024-11-24 21:02:03,252 [grpc-default-executor-1] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
	at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
	at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
	at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)

auth.json File

{
  "oAuthConfig": {
    "clientId": "###",
    "clientSecret": "###",
    "redirectUrl": "###",
    "authorityUrl": "###",
    "scope": "openid profile email",
    "jwtClaims": {
      "userName": "name"
    }
  },
  "ldapConfig": {
    "connectionMode": "PLAIN",
    "servers": [
      {
        "hostname": "###",
        "port": ###
      }
    ],
    "names": {
      "bindDN": "CN=Dremio Development Account,OU=Service,OU=UserObjects,DC=###,DC=###,DC=###",
      "bindPassword": "###",
      "baseDN": "DC=###,DC=###,DC=###",
      "userFilter": "objectClass=user",
      "userAttributes": {
        "baseDNs": [
          "DC=###,DC=###,DC=###"
        ],
        "searchScope": "SUB_TREE",
        "id": "cn",
        "firstname": "givenName",
        "lastname": "sn",
        "email": "mail"
      },
      "groupDNs": [
	 "CN={0},OU=GroupObjects,DC=###,DC=###,DC=###"
      ],
      "groupMembership": "memberOf",
      "groupFilter": "(objectClass=group)",
      "autoAdminFirstUser": true
    }
  }
}

Questions:

Is there a way to make Dremio explicitly log the values it retrieves for id, firstname, lastname, and email to verify they are not null?
Could the groupDNs or groupMembership configuration cause this issue?
Any pointers on how to resolve the NullPointerException or further debug this issue?

Any advice or insights would be greatly appreciated. Thanks in advance for your help!

balaji.ramaswamy · November 26, 2024, 4:33am

@lewis133

Can you please add below logger also and retry?

<logger name="com.dremio.extusr.ExternalUserGroupService">
<level value="debug"/>
</logger>

From outside Dremio if you send the belo 2 requests from something like ldapsearch does it return the expected result?

lewis133 · November 26, 2024, 1:56pm

Hi @balaji.ramaswamy ,

I am already using the following in the logback.xml:

<logger name="org.apache.hadoop">
    <level value="${dremio.log.level:-debug}"/>
  </logger>

  <logger name="com.dremio.extusr">
    <level value="trace"/>
  </logger>

  <logger name="com.dremio.service.roles">
    <level value="trace"/>
  </logger>
  <logger name="com.dremio.exec.store.ldap" level="DEBUG"/>
  <logger name="com.unboundid.ldap" level="trace"/>
  <logger name="com.dremio.service.usergroup" level="trace"/>
  <logger name="com.dremio.dac.daemon" level="trace"/>
  <logger name="com.dremio.extusr.ldap" level="trace"/>
  <logger name="com.dremio.extusr.ExternalUserGroupService" level="trace"/>

  <root>
    <level value="trace"/>
    <appender-ref ref="CONSOLE"/>
  </root>

Yes I have been using ldapsearch to make sure everything is accessible from Dremio and from what I can tell, this configuration should work. In the log its showing that its successfully finding the user and printing its LDAP groups, but then immediately failing after that. From the error I can’t tell where or why it is failing. My only guess is that it’s retrieving the values in this section properly which would be the next logical step:

"userAttributes": {
        "baseDNs": [
          "DC=###,DC=###,DC=###"
        ],
        "searchScope": "SUB_TREE",
        "id": "cn",
        "firstname": "givenName",
        "lastname": "sn",
        "email": "mail"
      },

I’ve queried LDAP with ldapsearch and was able to retrieve those values no problem.

Here is the full relevant section from the log:

2024-11-26 14:02:22,549 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Finishing OAuth authorization flow for session [###]...
2024-11-26 14:02:23,442 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Completing the OAuth authorization flow for session [###] took 893 ms
2024-11-26 14:02:23,721 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Looking up user using filter [(&(cn=Jordan Lewis)(objectClass=user))] with base [DC=###,DC=###,DC=com] and scope [SUB] took 37 ms
2024-11-26 14:02:23,725 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 2 ms
2024-11-26 14:02:23,727 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Global,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,728 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Team,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,729 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,730 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,731 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,733 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,734 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,735 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,736 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,736 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,737 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,738 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,739 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,740 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,741 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,742 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,743 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,744 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,745 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,752 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - OAuth authorization flow for session [###] resolved to user [Jordan Lewis] and their token expires at ###
2024-11-26 14:02:23,778 [qtp1284345216-264] INFO  audit.logger - Audit
2024-11-26 14:02:23,792 [grpc-default-executor-2] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
	at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
	at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
	at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.roles.LocalRolesServiceImpl.getRolesForUserToExpand(LocalRolesServiceImpl.java:294)
	at com.dremio.service.roles.RolesServiceImpl.getRolesForUser(RolesServiceImpl.java:908)
	at com.dremio.service.roles.common.proto.RolesServiceGrpc$MethodHandlers.invoke(RolesServiceGrpc.java:1729)
	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at io.opentracing.contrib.grpc.TracingServerInterceptor$2.onHalfClose(TracingServerInterceptor.java:231)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at com.dremio.service.grpc.ContextualizedServerInterceptor$1.lambda$onHalfClose$0(ContextualizedServerInterceptor.java:74)
	at com.dremio.context.RequestContext.run(RequestContext.java:103)
	at com.dremio.service.grpc.ContextualizedServerInterceptor$1.onHalfClose(ContextualizedServerInterceptor.java:74)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at io.grpc.util.TransmitStatusRuntimeExceptionInterceptor$1.onHalfClose(TransmitStatusRuntimeExceptionInterceptor.java:74)
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:356)
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:861)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
2024-11-26 14:02:23,794 [qtp1284345216-264] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
	at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
	at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
	at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:23)
	at com.dremio.services.accesscontrol.PrivilegeEnforcerImpl$Factory.get(PrivilegeEnforcerImpl.java:227)
	at com.dremio.dac.daemon.EnterpriseDACDaemonModule.lambda$build$23(EnterpriseDACDaemonModule.java:1363)
	at com.dremio.dac.resource.EnterpriseLogin.lambda$checkProjectPrivilege$1(EnterpriseLogin.java:176)
	at com.dremio.context.RequestContext.call(RequestContext.java:121)
	at com.dremio.dac.resource.EnterpriseLogin.checkProjectPrivilege(EnterpriseLogin.java:173)
	at com.dremio.dac.resource.EnterpriseLogin.createLoginSession(EnterpriseLogin.java:134)
	at com.dremio.dac.resource.EnterpriseSSOResource.finish(EnterpriseSSOResource.java:111)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.dremio.dac.server.EnterpriseContextualizedResourceMethodInvocationHandlerProvider.lambda$create$0(EnterpriseContextualizedResourceMethodInvocationHandlerProvider.java:55)
	at com.dremio.context.RequestContext.call(RequestContext.java:121)
	at com.dremio.dac.server.EnterpriseContextualizedResourceMethodInvocationHandlerProvider.lambda$create$1(EnterpriseContextualizedResourceMethodInvocationHandlerProvider.java:55)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:397)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:349)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:379)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:312)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
	at com.dremio.dac.server.GenericResponseHeadersFilter.doFilter(GenericResponseHeadersFilter.java:44)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
	at com.dremio.dac.server.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:71)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:722)
	at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.base/java.lang.Thread.run(Thread.java:829)

Here is what I get when I use ldapsearch to query for those values directly:

root@ns-atx1-01:/home/jlewis/git/dremio-poc# ldapsearch -x -LLL \
  -H ldap://###.###.###.com:3268 \
  -D "CN=###,OU=Service,OU=UserObjects,DC=###,DC=###,DC=com" \
  -w "###" \
  -b "DC=###,DC=###,DC=com" \
  "(&(objectClass=user)(cn=Jordan Lewis))" \
  cn mail givenName sn memberOf
dn: CN=Jordan Lewis,OU=###,OU=UserObjects,DC=###,DC=###,DC=com
cn: Jordan Lewis
sn: Lewis
givenName: Jordan
memberOf: CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com
memberOf: CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com
memberOf: CN=###,OU=Team,OU=GroupObjects,DC=###,DC=###,DC=com
memberOf: CN=###,OU=Global,OU=GroupObjects,DC=###,DC=###,DC=com
mail: ###@###.com

root@ns-atx1-01:/home/jlewis/git/dremio-poc#

I have noticed that the ldapsearch results only show 4 groups, whereas the dremio log shows many more. Guessing Dremio’s ldap query is different than what I’m using to test with ldapsearch

lewis133 · December 3, 2024, 9:59pm

Hi @balaji.ramaswamy, checking in to see if you have had a chance to look at this. Thank you!

balaji.ramaswamy · December 5, 2024, 6:57am

@lewis133 Apologies, I have been a little busy last 10 days. It seems like you are hitting a NullPointerException, let me investigate this stack and get back to you

balaji.ramaswamy · December 5, 2024, 9:10pm

@lewis133 response form one of our engineers

From a high level, it looks like it might be a config issue between OAuth and the LDAP provider. For mode “oauth+ ldap,” we have a two-step process.
Step 1: we authenticate the user using OAuth on the web browser and establish the dremio user from the ID Token (using the userName claim mapping).
Step 2: After establishing the username, we query the configured LDAP to retrieve user details and groups. Failing to do so will result in authentication failure. Note; the user filters here will use the dremio established username in step 1.

For this to work, we should ensure that the claim mapping you provide for the userName in oauth.json can be used to search users from the configured LDAP.

lewis133 · December 5, 2024, 9:44pm

@balaji.ramaswamy From the log I provided above it looks like it is successfully querying LDAP and retrieving my user’s groups:

2024-11-26 14:02:22,549 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Finishing OAuth authorization flow for session [###]...
2024-11-26 14:02:23,442 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Completing the OAuth authorization flow for session [###] took 893 ms
2024-11-26 14:02:23,721 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Looking up user using filter [(&(cn=Jordan Lewis)(objectClass=user))] with base [DC=###,DC=###,DC=com] and scope [SUB] took 37 ms
2024-11-26 14:02:23,725 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 2 ms
2024-11-26 14:02:23,727 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Global,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,728 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Team,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,729 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,730 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,731 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,733 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,734 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,735 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,736 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,736 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,737 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,738 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,739 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,740 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,741 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,742 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,743 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,744 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,745 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,752 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - OAuth authorization flow for session [###] resolved to user [Jordan Lewis] and their token expires at ###
2024-11-26 14:02:23,778 [qtp1284345216-264] INFO  audit.logger - Audit
2024-11-26 14:02:23,792 [grpc-default-executor-2] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
	at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
	at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
	at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.roles.LocalRolesServiceImpl.getRolesForUserToExpand(LocalRolesServiceImpl.java:294)
	at com.dremio.service.roles.RolesServiceImpl.getRolesForUser(RolesServiceImpl.java:908)
	at com.dremio.service.roles.common.proto.RolesServiceGrpc$MethodHandlers.invoke(RolesServiceGrpc.java:1729)
	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at io.opentracing.contrib.grpc.TracingServerInterceptor$2.onHalfClose(TracingServerInterceptor.java:231)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at com.dremio.service.grpc.ContextualizedServerInterceptor$1.lambda$onHalfClose$0(ContextualizedServerInterceptor.java:74)
	at com.dremio.context.RequestContext.run(RequestContext.java:103)
	at com.dremio.service.grpc.ContextualizedServerInterceptor$1.onHalfClose(ContextualizedServerInterceptor.java:74)
	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
	at io.grpc.util.TransmitStatusRuntimeExceptionInterceptor$1.onHalfClose(TransmitStatusRuntimeExceptionInterceptor.java:74)
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:356)
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:861)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
2024-11-26 14:02:23,794 [qtp1284345216-264] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
	at java.base/java.util.Objects.requireNonNull(Objects.java:222)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
	at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
	at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
	at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
	at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
	at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
	at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:23)
	at com.dremio.services.accesscontrol.PrivilegeEnforcerImpl$Factory.get(PrivilegeEnforcerImpl.java:227)
	at com.dremio.dac.daemon.EnterpriseDACDaemonModule.lambda$build$23(EnterpriseDACDaemonModule.java:1363)
	at com.dremio.dac.resource.EnterpriseLogin.lambda$checkProjectPrivilege$1(EnterpriseLogin.java:176)
	at com.dremio.context.RequestContext.call(RequestContext.java:121)
	at com.dremio.dac.resource.EnterpriseLogin.checkProjectPrivilege(EnterpriseLogin.java:173)
	at com.dremio.dac.resource.EnterpriseLogin.createLoginSession(EnterpriseLogin.java:134)
	at com.dremio.dac.resource.EnterpriseSSOResource.finish(EnterpriseSSOResource.java:111)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.dremio.dac.server.EnterpriseContextualizedResourceMethodInvocationHandlerProvider.lambda$create$0(EnterpriseContextualizedResourceMethodInvocationHandlerProvider.java:55)
	at com.dremio.context.RequestContext.call(RequestContext.java:121)
	at com.dremio.dac.server.EnterpriseContextualizedResourceMethodInvocationHandlerProvider.lambda$create$1(EnterpriseContextualizedResourceMethodInvocationHandlerProvider.java:55)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:397)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:349)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:379)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:312)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
	at com.dremio.dac.server.GenericResponseHeadersFilter.doFilter(GenericResponseHeadersFilter.java:44)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
	at com.dremio.dac.server.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:71)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:722)
	at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.base/java.lang.Thread.run(Thread.java:829)

Side note: I have used both the user’s cn and email/mail attributes as the userName/id values to debug, I get the same results for both. My name and email are both unique in the SSO/LDAP systems, i.e. there is no one else at my company with my same name/email.

lewis133 · January 2, 2025, 5:03pm

Hey @balaji.ramaswamy wanted to follow up on this post-holidays. Have you had a chance to review the above?

balaji.ramaswamy · January 8, 2025, 6:06am

@lewis133 Yes, let me get back to you on this

lewis133 · January 21, 2025, 5:23pm

Hey @balaji.ramaswamy checking in to see if you have found any leads for this issue

balaji.ramaswamy · February 11, 2025, 7:49pm

@lewis133

Did you create an EXTERNAL user before OAuth + LDAP was configured for the LDAP user that you are trying to log in as?

Thanks
Bali

balaji.ramaswamy · February 11, 2025, 8:04pm

In other words, is there a local user with the same name too? Able to send us the output of

select * from sys.users where username = '<login-user-name>'

lewis133 · February 11, 2025, 8:07pm

I initially configured OAuth and was able to sign in with my SSO account which I see is marked as external in the accounts list here:

Then I moved onto configuring the LDAP. When I sign in with this account, in the Dremio log I see it printing all the LDAP groups this account belongs to but then fails with the null pointer exception

lewis133 · February 11, 2025, 8:10pm

Just saw your second message, there is one account with this username:

balaji.ramaswamy · February 11, 2025, 8:14pm

@lewis133, Can you try deleting the user from the UI? go to settings-users, the user will exist as EXTERNAL and delete that user and retry

lewis133 · February 11, 2025, 11:11pm

I’m not entirely sure what changed here, possibly me deleting the account like you suggested, but I am getting a new error now.

It is not giving the NPE anymore, but it is saying could not find external role for group <ldap group here>.

I already have a manually created role in dremio with the same name as one of the LDAP groups I am in, which I believe is needed in order for the LDAP <—> Dremio Role mapping to happen, and I am assuming what the log is saying is that its trying to find existing roles in dremio with the same name as the LDAP groups but its failing.

The word external makes me wonder if its trying to fetch the groups I am in through SSO as well though, which is not what I am intending.

I abbreviated the log below with ... for brevity

2025-02-11 23:00:26,764 [qtp698080447-351] TRACE c.d.extusr.oauth.OAuthServiceImpl - Starting OAuth authorization flow for session [########-####-####-####-############] with requestState [%7B%22sessionId%22%3A%22########-####-####-####-############%22%2C%22uiRedirectUrl%22%3A%22%22%2C%22stateID%22%3Anull%7D] and redirect url [https://dremio-poc.########.com/sso]
2025-02-11 23:00:28,878 [qtp698080447-335] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Finishing OAuth authorization flow for session [########-####-####-####-############]...
2025-02-11 23:00:29,520 [qtp698080447-335] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Completing the OAuth authorization flow for session [########-####-####-####-############] took 642 ms
2025-02-11 23:00:29,798 [qtp698080447-335] DEBUG c.d.extusr.ldap.LdapUserProvider - Looking up user using filter [(&(cn=##### #####)(objectClass=user))] with base [DC=####,DC=########,DC=com] and scope [SUB] took 41 ms
2025-02-11 23:00:29,806 [qtp698080447-335] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=########,OU=Access,OU=GroupObjects,DC=####,DC=########,DC=com] took 6 ms
2025-02-11 23:00:29,814 [qtp698080447-335] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=########,OU=Global,OU=GroupObjects,DC=####,DC=########,DC=com] took 7 ms
2025-02-11 23:00:29,820 [qtp698080447-335] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=########,OU=Team,OU=GroupObjects,DC=####,DC=########,DC=com] took 5 ms
...
2025-02-11 23:00:29,952 [qtp698080447-335] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=########,OU=Access,OU=GroupObjects,DC=####,DC=########,DC=com] took 6 ms
2025-02-11 23:00:30,075 [qtp698080447-335] DEBUG c.d.extusr.oauth.OAuthServiceImpl - OAuth authorization flow for session [########-####-####-####-############] resolved to user [##### #####] and their token expires at ########### with a refresh token provided
2025-02-11 23:00:30,191 [qtp698080447-335] INFO  audit.logger - Audit
2025-02-11 23:00:30,234 [qtp698080447-335] INFO  c.d.s.n.c.CatalogEventMessagePublisherProviderImpl - Disabling CatalogEventMessagePublisher.
2025-02-11 23:00:30,251 [grpc-default-executor-2] DEBUG c.d.s.roles.LocalRolesServiceImpl - could not find external role for group '########'
2025-02-11 23:00:30,252 [grpc-default-executor-2] DEBUG c.d.s.roles.LocalRolesServiceImpl - could not find external role for group '########'
2025-02-11 23:00:30,252 [grpc-default-executor-2] DEBUG c.d.s.roles.LocalRolesServiceImpl - could not find external role for group '########'
...
2025-02-11 23:00:30,258 [grpc-default-executor-2] DEBUG c.d.s.roles.LocalRolesServiceImpl - could not find external role for group '########'

balaji.ramaswamy · February 12, 2025, 10:16pm

@lewis133 Can you please try and delete the role under settings-role- and see if it works?

lewis133 · February 12, 2025, 10:36pm

Same error after deleting the role and signing in unfortunately

Topic		Replies	Views
Reflection refresh is failing with NullPointerException in Dremio 24.2.2	8	573	January 4, 2024
Error SSO for dremio connexion	8	1447	March 25, 2022
Null Pointer Exception	1	1382	September 5, 2021
Unable to Connect Dremio with Azure Data Lake Storage Gen1	10	2383	July 22, 2020
Unknown error when trying to read some data in Elasticsearch	1	1486	March 14, 2019

NullPointerException During LDAP User Authentication in Dremio with "oauth+ldap" coordinator.web.auth.type

Related topics