Custom Spring UserDetailsService not called

I'm trying to add some custom data for currently logged in user, so I found that I can implement my own UserDetailsService and just plug it in Spring, but it's never called, I always get Principal as just username string.

I have my implementation of UserDetailsService:

@Service
public class UserDetailServiceImpl implements UserDetailsService {

  @Override
  public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {
    ...
  }
}

my implementation of UserDetails:

import org.springframework.security.core.userdetails.User;

public class LoggedInUser extends User {
...
}

and tried setting it in config (SecurityConfiguration) multiple ways:

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityConfiguration extends WebSecurityConfigurerAdapter {
      @Autowired
      private CustomAuthenticationProvider customAuthProvider;
      @Autowired
      private UserDetailsService userDetailServiceImpl;
      @Override
      protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.authenticationProvider(customAuthProvider);
        auth.userDetailsService(userDetailServiceImpl).passwordEncoder(passwordService.getPasswordEncoder());
      }
...
}

or

@Autowired
  protected void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
    auth.authenticationProvider(customAuthProvider);
    auth.userDetailsService(userDetailServiceImpl).passwordEncoder(passwordService.getPasswordEncoder());
  }

or

@Override
  @Bean
  public UserDetailsService userDetailsService() {
    return new UserDetailServiceImpl();
  }

and nothing works... I tried retrieving user info multiple ways:

1. in controller with @AuthenticationPrincipal where I get null

2. in service (I get invalid cast error):

   Authentication authentication = SecurityContextHolder.getContext().getAuthentication(); (LoggedInUser) authentication.getPrincipal()

Any idea why it's not working? Is my impl class being overriden by default one somewhere else? I tried to look in the logs (logging.level.org.springframework.security=TRACE) but no luck :/ I can login, that works fine, just principal data is always only username String and not my class.

ILya Cyclone's comment was what was the key to figuring out what was wrong, after checking CustomAuthenticationProvider again I noticed that when returning data you can specify Principal that does not need to be only username, but you can add your custom UserDetails there and because of that custom UserDetailsService is not called, which was my mistake in assuming that it will be always called.

So in the end this is enough to have custom AuthenticationProvider with custom UserDetails:

@Service
public class CustomAuthenticationProvider implements AuthenticationProvider {

  @Autowired
  private UserService userService;

  @Autowired
  private PasswordService passwordService;

  @Override
  public Authentication authenticate(Authentication auth) throws AuthenticationException {
    String username = auth.getName();
    String password = auth.getCredentials().toString();

    User user = userService.getUserWithPermissionsByName(username);
    if (user == null) {
      throw new BadCredentialsException("invalid_username_or_pass");
    }

    if (!passwordService.passwordsMatch(password, user.getPassword())) {
      throw new BadCredentialsException("invalid_username_or_pass");
    }

    String[] permissions = user.getPermissions().stream().map((p) -> p.getName()).toArray(String[]::new);
    List<GrantedAuthority> grantedAuths = AuthorityUtils.createAuthorityList(permissions);
    return new UsernamePasswordAuthenticationToken(new LoggedInUser(user.getName(), user.getPassword(), true, true, true, true, grantedAuths, user.getId()),
        password, grantedAuths);
  }

  @Override
  public boolean supports(Class<?> auth) {
    return auth.equals(UsernamePasswordAuthenticationToken.class);
  }
}