kerberoshortonworks-data-platformambariapache-knox

Why does ambari is showing this kerberos authentication error : AmbariAuthToLocalUserDetailsService


Recently ambari server logs are showing some warnings, We didn't identify any issue yet but logs became unreadable (50 lines/s)

/var/log/ambari-server/ambari-server.log

02 Jul 2021 18:43:52,514  INFO [ambari-client-thread-792188] AmbariAuthToLocalUserDetailsService:109 - Translated knox/<knox_gateway>@<REALM> to knox using auth-to-local rules during Kerberos authentication.
02 Jul 2021 18:43:52,515  WARN [ambari-client-thread-792188] AmbariAuthToLocalUserDetailsService:143 - Failed find user account for user with username of knox during Kerberos authentication.
02 Jul 2021 18:43:52,516  WARN [ambari-client-thread-792188] AmbariKerberosAuthenticationFilter:149 - Negotiate Header was invalid: Negotiate YIIDl...
org.springframework.security.core.userdetails.UsernameNotFoundException: Failed find user account for user with username of knox during Kerberos authentication.
        at org.apache.ambari.server.security.authentication.kerberos.AmbariAuthToLocalUserDetailsService.createUser(AmbariAuthToLocalUserDetailsService.java:144)
        at org.apache.ambari.server.security.authentication.kerberos.AmbariAuthToLocalUserDetailsService.loadUserByUsername(AmbariAuthToLocalUserDetailsService.java:110)
        at org.springframework.security.kerberos.authentication.KerberosServiceAuthenticationProvider.authenticate(KerberosServiceAuthenticationProvider.java:66)
        at org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:156)
        at org.springframework.security.kerberos.web.authentication.SpnegoAuthenticationProcessingFilter.doFilter(SpnegoAuthenticationProcessingFilter.java:145)
        at org.apache.ambari.server.security.authentication.kerberos.AmbariKerberosAuthenticationFilter.doFilter(AmbariKerberosAuthenticationFilter.java:167)
        at org.apache.ambari.server.security.authentication.AmbariDelegatingAuthenticationFilter.doFilter(AmbariDelegatingAuthenticationFilter.java:120)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.apache.ambari.server.security.authorization.AmbariUserAuthorizationFilter.doFilter(AmbariUserAuthorizationFilter.java:91)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
        at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
        at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
        at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:125)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
        at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1478)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:427)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
        at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:212)
        at org.apache.ambari.server.controller.AmbariHandlerList.processHandlers(AmbariHandlerList.java:201)
        at org.apache.ambari.server.controller.AmbariHandlerList.handle(AmbariHandlerList.java:139)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
        at org.eclipse.jetty.server.Server.handle(Server.java:370)
        at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
        at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:973)
        at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1035)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:641)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:231)
        at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)

/var/log/ambari-server/ambari-audit.log

2021-07-02T19:01:16.881+0200, User(null), RemoteIp(xxx.xxx.xxx.xxx), Operation(User login), Roles(
), Status(Failed), Reason(Failed find user account for user with username of knox during Kerberos authentication.)

Known issue : https://issues.apache.org/jira/browse/AMBARI-19767

Ambari version : 2.6.2.2 HDP Version : HDP-2.6.5.1100


Solution

  • The root cause of this issue is when the kerberos authentication is enabled for ambari the logout feature will not work any more.

    disabling kerberos authentication on ambari server resolved the issue

    authentication.kerberos.enabled=false