How to use authorization by default without [Authorize] in ASP.NET Core 5

I have an ASP.NET Core 5 application.

I've got authorization working with the [Authorize] attribute above the controller endpoint.

How can I make it use authorization without having to use the [Authorize] attribute? Here is how I set up my authorization:

I have this in ConfigureServices:

public void ConfigureServices(IServiceCollection services)
{...
        FirebaseApp.Create(new AppOptions()
        {
            Credential = GoogleCredential.FromFile("firebase_admin_sdk.json"),
        });

        services
        .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
        .AddJwtBearer(options =>
        {
            options.Authority = "https://securetoken.google.com/vepo-xxx";
            options.TokenValidationParameters = new TokenValidationParameters
            {
                ValidateIssuer = true,
                ValidIssuer = "https://securetoken.google.com/vepo-xxx",
                ValidateAudience = true,
                ValidAudience = "vepo-xxx",
                ValidateLifetime = true
            };
        });
...}

And in I Configure have:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env, VepoContext context, ISearchIndexService searchIndexService)
{...
    app.UseAuthentication();
    app.UseAuthorization();
...}

See the docs for Require authenticated users. You can add a RequireAuthenticatedUser() to all controllers:

public void ConfigureServices(IServiceCollection services)
{

    ...

    services.AddControllers(config =>
    {
        // using Microsoft.AspNetCore.Mvc.Authorization;
        // using Microsoft.AspNetCore.Authorization;
        var policy = new AuthorizationPolicyBuilder()
                         .RequireAuthenticatedUser()
                         .Build();
        config.Filters.Add(new AuthorizeFilter(policy));
    });