Where should roles and missions/ACLs be managed when building an authorization server?

I’m trying to build my own Spring Authorization Server (for learning purposes and possibly to use across multiple projects in the future).

I’m already familiar with Spring Security and resource server concepts, but I’m confused about the architecture for roles and permissions (ACLs).

My goal:

• Create an OIDC/OAuth2 Authorization Server that issues JWT + refresh tokens.

• Each client application may define different roles and permissions for its users.

• In the future, I want to support fine-grained ACLs (like per-resource ownership, e.g., User A can view image X but not Y).

Now I’m confused about the best place to store and enforce roles/permissions/ACLs:

1. Authorization Server Approach (centralized, like Keycloak):

   • Store a mapping of Client → Roles → Permissions in the Authorization Server DB.

   • Issue JWTs with embedded roles/ACL claims.

   • All client apps/resource servers rely on these claims.

   • Centralized management, but could become complex if ACLs are very fine-grained (e.g., per resource ownership).

2. Resource Server Approach

   • Authorization server only handles authentication (tokens, identity).

   • Each resource server/client API manages its own roles, permissions, and ACLs in its own database.

   • Allows domain-specific permissions but makes cross-service role management harder.

What I want to understand:

• Trade-offs of centralized ACL in Authorization Server vs distributed ACL in resource servers

• Where is the recommended place to manage roles and ACLs in this kind of setup?

• Would it make sense to reference the client_id in my Authorization Server DB and maintain separate role/permission sets for each client, or is that overcomplicating it?

In what you describe, there is a confusion between roles, which are user attributes, and resource access.

A centralized resource access decision service is a disaster in terms of:

• performance: adds the latency of a round trip to an external service
• separation of concerns: when the decision depends on the resource themselves, the centralized service must know about some of these resources atttributes and how they realte to each other (what an image is and how it relates to users)

I define user roles on a per-client basis, and configure the authorization-server to put in tokens only the roles for the client requesting the tokens. Then, resource servers can take access control decision based on the access token claims (mainly user subject and roles) and the accessed resource, without the need of any other service.

Keycloak's handles roles at the client level, Keycloak's authorization service is optional and deactivated by default, and Spring Security SpEL allows referencing secured method arguments.

Let's say that we have an ImageRepo that can store ImageEntity instances based on their ID.

private final ImageRepo imageRepo;

@GetMapping("/my-images")
@PreAuthorize("isAuthenticated() && hasAuthority('images.read')")
public List<ImageEntity> getMyImages(JwtAuthenticationToken oauth) {
  return imageRepo.findByAuthor(oauth.getName());
}

@PutMapping("/images/{id}")
@PreAuthorize("isAuthenticated() && #imageToUpdate.author == #oauth.name")
public void updateImage(
    @PathVariable("id") ImageEntity imageToUpdate, // auto-magically resolved by the imageRepo from the ImageEntity ID
    @RequestPart MultipartFile dto, 
    JwtAuthenticationToken oauth) {
  ...
}