Hi @balaji.ramaswamy ,
I am already using the following in the logback.xml:
<logger name="org.apache.hadoop">
<level value="${dremio.log.level:-debug}"/>
</logger>
<logger name="com.dremio.extusr">
<level value="trace"/>
</logger>
<logger name="com.dremio.service.roles">
<level value="trace"/>
</logger>
<logger name="com.dremio.exec.store.ldap" level="DEBUG"/>
<logger name="com.unboundid.ldap" level="trace"/>
<logger name="com.dremio.service.usergroup" level="trace"/>
<logger name="com.dremio.dac.daemon" level="trace"/>
<logger name="com.dremio.extusr.ldap" level="trace"/>
<logger name="com.dremio.extusr.ExternalUserGroupService" level="trace"/>
<root>
<level value="trace"/>
<appender-ref ref="CONSOLE"/>
</root>
Yes I have been using ldapsearch
to make sure everything is accessible from Dremio and from what I can tell, this configuration should work. In the log its showing that its successfully finding the user and printing its LDAP groups, but then immediately failing after that. From the error I can’t tell where or why it is failing. My only guess is that it’s retrieving the values in this section properly which would be the next logical step:
"userAttributes": {
"baseDNs": [
"DC=###,DC=###,DC=###"
],
"searchScope": "SUB_TREE",
"id": "cn",
"firstname": "givenName",
"lastname": "sn",
"email": "mail"
},
I’ve queried LDAP with ldapsearch
and was able to retrieve those values no problem.
Here is the full relevant section from the log:
2024-11-26 14:02:22,549 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Finishing OAuth authorization flow for session [###]...
2024-11-26 14:02:23,442 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - Completing the OAuth authorization flow for session [###] took 893 ms
2024-11-26 14:02:23,721 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Looking up user using filter [(&(cn=Jordan Lewis)(objectClass=user))] with base [DC=###,DC=###,DC=com] and scope [SUB] took 37 ms
2024-11-26 14:02:23,725 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 2 ms
2024-11-26 14:02:23,727 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Global,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,728 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Team,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,729 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,730 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,731 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,733 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,734 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,735 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,736 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,736 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,737 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,738 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,739 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,740 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,741 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,742 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com] took 1 ms
2024-11-26 14:02:23,743 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,744 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,745 [qtp1284345216-264] DEBUG c.d.extusr.ldap.LdapUserProvider - Fetching group entry with DN [CN=###,OU=Product,OU=Data,OU=GroupObjects,DC=###,DC=###,DC=com] took 0 ms
2024-11-26 14:02:23,752 [qtp1284345216-264] DEBUG c.d.extusr.oauth.OAuthServiceImpl - OAuth authorization flow for session [###] resolved to user [Jordan Lewis] and their token expires at ###
2024-11-26 14:02:23,778 [qtp1284345216-264] INFO audit.logger - Audit
2024-11-26 14:02:23,792 [grpc-default-executor-2] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:222)
at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)
at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
at com.dremio.service.roles.LocalRolesServiceImpl.getRolesForUserToExpand(LocalRolesServiceImpl.java:294)
at com.dremio.service.roles.RolesServiceImpl.getRolesForUser(RolesServiceImpl.java:908)
at com.dremio.service.roles.common.proto.RolesServiceGrpc$MethodHandlers.invoke(RolesServiceGrpc.java:1729)
at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at io.opentracing.contrib.grpc.TracingServerInterceptor$2.onHalfClose(TracingServerInterceptor.java:231)
at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at com.dremio.service.grpc.ContextualizedServerInterceptor$1.lambda$onHalfClose$0(ContextualizedServerInterceptor.java:74)
at com.dremio.context.RequestContext.run(RequestContext.java:103)
at com.dremio.service.grpc.ContextualizedServerInterceptor$1.onHalfClose(ContextualizedServerInterceptor.java:74)
at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at io.grpc.util.TransmitStatusRuntimeExceptionInterceptor$1.onHalfClose(TransmitStatusRuntimeExceptionInterceptor.java:74)
at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:356)
at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:861)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2024-11-26 14:02:23,794 [qtp1284345216-264] DEBUG c.d.extusr.ExternalUserGroupService - Fail to load user: null
java.lang.NullPointerException: null
at java.base/java.util.Objects.requireNonNull(Objects.java:222)
at com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2648)
at com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:112)
at com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:58)
at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:214)
at com.dremio.extusr.ExternalUserGroupService.getUser(ExternalUserGroupService.java:229)
at com.dremio.extusr.oauthldap.OAuthLDAPUserGroupService.getUser(OAuthLDAPUserGroupService.java:87)
at com.dremio.service.usergroup.ProxyingUserGroupService.getUserInternal(ProxyingUserGroupService.java:160)
at com.dremio.service.usergroup.ProxyingUserGroupService.getUser(ProxyingUserGroupService.java:117)
at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:77)
at com.dremio.service.usergroup.UserGroupServiceWrapper.getUser(UserGroupServiceWrapper.java:23)
at com.dremio.services.accesscontrol.PrivilegeEnforcerImpl$Factory.get(PrivilegeEnforcerImpl.java:227)
at com.dremio.dac.daemon.EnterpriseDACDaemonModule.lambda$build$23(EnterpriseDACDaemonModule.java:1363)
at com.dremio.dac.resource.EnterpriseLogin.lambda$checkProjectPrivilege$1(EnterpriseLogin.java:176)
at com.dremio.context.RequestContext.call(RequestContext.java:121)
at com.dremio.dac.resource.EnterpriseLogin.checkProjectPrivilege(EnterpriseLogin.java:173)
at com.dremio.dac.resource.EnterpriseLogin.createLoginSession(EnterpriseLogin.java:134)
at com.dremio.dac.resource.EnterpriseSSOResource.finish(EnterpriseSSOResource.java:111)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.dremio.dac.server.EnterpriseContextualizedResourceMethodInvocationHandlerProvider.lambda$create$0(EnterpriseContextualizedResourceMethodInvocationHandlerProvider.java:55)
at com.dremio.context.RequestContext.call(RequestContext.java:121)
at com.dremio.dac.server.EnterpriseContextualizedResourceMethodInvocationHandlerProvider.lambda$create$1(EnterpriseContextualizedResourceMethodInvocationHandlerProvider.java:55)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:397)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:349)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:379)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:312)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
at com.dremio.dac.server.GenericResponseHeadersFilter.doFilter(GenericResponseHeadersFilter.java:44)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at com.dremio.dac.server.SecurityHeadersFilter.doFilter(SecurityHeadersFilter.java:71)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:722)
at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:54)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
Here is what I get when I use ldapsearch
to query for those values directly:
root@ns-atx1-01:/home/jlewis/git/dremio-poc# ldapsearch -x -LLL \
-H ldap://###.###.###.com:3268 \
-D "CN=###,OU=Service,OU=UserObjects,DC=###,DC=###,DC=com" \
-w "###" \
-b "DC=###,DC=###,DC=com" \
"(&(objectClass=user)(cn=Jordan Lewis))" \
cn mail givenName sn memberOf
dn: CN=Jordan Lewis,OU=###,OU=UserObjects,DC=###,DC=###,DC=com
cn: Jordan Lewis
sn: Lewis
givenName: Jordan
memberOf: CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com
memberOf: CN=###,OU=Access,OU=GroupObjects,DC=###,DC=###,DC=com
memberOf: CN=###,OU=Team,OU=GroupObjects,DC=###,DC=###,DC=com
memberOf: CN=###,OU=Global,OU=GroupObjects,DC=###,DC=###,DC=com
mail: ###@###.com
root@ns-atx1-01:/home/jlewis/git/dremio-poc#
I have noticed that the ldapsearch
results only show 4 groups, whereas the dremio log shows many more. Guessing Dremio’s ldap query is different than what I’m using to test with ldapsearch