Spring Boot Oauth2 Resource Server UserDetailsService

Trying to get a UserDetailsService working for an oauth2 resource server I set up. I'm able to successfully authenticate the jwt, but nothing I do seems to get it to call the loadUserByUsername method. This originally was using SAML and it was working, but now I've cut over to Oauth2 and I can't get it working.

     @Service
     public class OauthUsersDetailsServiceImpl implements UserDetailsService{
         @Override
         public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {
             //some user loading junk here - this is never called
         }
     }

     @Configuration
        @EnableGlobalMethodSecurity(prePostEnabled = true)
        @EnableWebSecurity
         public class SecurityConfig extends WebSecurityConfigurerAdapter {
            
            @Override
            protected void configure(HttpSecurity http) throws Exception
            {
                //test key for now
                SecretKeySpec key = new SecretKeySpec("private key0000000000000000000000000000000".getBytes(), "HMACSHA256");
                

                http
                    .authorizeRequests()
                    .antMatchers(/*some endpoints im excluding from auth - this all works*/)
                    .permitAll().and()
                    .authorizeRequests()
                    .anyRequest().authenticated().and()
                    .oauth2ResourceServer().jwt().decoder(NimbusJwtDecoder.withSecretKey(key).build());
            }
         }

I found with google that I could just register the class as a bean with @service and spring would just pick it up, but it's not working. I also tried adding it through the AuthenticationManagerBuilder, but that didn't work either. My guess is that the jwt side of this has its own UserDetailsService that its implemented and is taking priority over mine. That said, what is the proper way to get mine to call, or is it better to somehow call my user loading logic manually after authentication is complete and overwrite the Principal object? I need this to happen before an endpoint is called so PreAuthorize can check for the roles that were loaded by the UserDetailsService.

Figured it out. Hopefully this will help anyone that comes across the same problem. I had to add a custom filter into the chain to call my user details service and overwrite the context:

public class Oauth2AuthorizationFilter extends GenericFilterBean {

        @Autowired
        private OauthUsersDetailsServiceImpl oauthUsersDetailsServiceImpl;
      
      public Oauth2AuthorizationFilter (OauthUsersDetailsServiceImpl userDetailsService) {
        this.oauthUsersDetailsServiceImpl = userDetailsService;
      }
      
      
      @Override
      public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
          throws IOException, ServletException {

        SecurityContext context = SecurityContextHolder.getContext();
        if(context.getAuthentication() != null && !(context.getAuthentication().getPrincipal() instanceof Users)) {
          
          UserDetails user = oauthUsersDetailsServiceImpl.loadUserByUsername(((Jwt)context.getAuthentication().getPrincipal()).getClaimAsString("user_name")); 
          UsernamePasswordAuthenticationToken authentication = new UsernamePasswordAuthenticationToken(user, null, user.getAuthorities());
          context.setAuthentication(authentication);
        }
        
        chain.doFilter(request, response);
      }

    }

@Override
        protected void configure(HttpSecurity http) throws Exception
        {
            //test key for now
            SecretKeySpec key = new SecretKeySpec("private key0000000000000000000000000000000".getBytes(), "HMACSHA256");
            
            http.authorizeRequests().antMatchers(/*bunch of junk...*/).permitAll().and().authorizeRequests().anyRequest().authenticated().and()
                .oauth2ResourceServer().jwt().decoder(NimbusJwtDecoder.withSecretKey(key).build());
            
            http.addFilterAfter(jwtAuthTokenFilterBean(), SwitchUserFilter.class);

        }

That finally did what I needed