why offline_access scope is needed to request refresh token in IdentityServer (OAuth2)?

I have to set AllowOfflineAccess = true to get refresh token

public static class Config
{
    public static IEnumerable<Client> Clients =>
        new Client[] 
        {
            new Client()
            {
                AllowOfflineAccess = true,
                // ....
            }
        }
}

according to OpenID specification https://openid.net/specs/openid-connect-core-1_0.html#OfflineAccess

offline_access OPTIONAL. This scope value requests that an OAuth 2.0 Refresh Token be issued that can be used to obtain an Access Token that grants access to the End-User's UserInfo Endpoint even when the End-User is not present (not logged in).

I don't want the "offline" feature, I want "online" feature where users remains login in idp, and when user can still access the resource after the access token expires and then refresh token kicks in. So why obtaining a refresh token has to do with "offline"?

Firstly, this is documented as neccessary. https://hts.readthedocs.io/en/latest/topics/grant_types.html?highlight=offline_access#refresh-tokens.

Since there are seldom doumentation about ids4, you could reference this explaination about Azure Oauth2 "offline_access"

The access token is usually valid for around one hour. At that point, your app needs to redirect the user back to the /authorize endpoint to request a new authorization code. During this redirect and depending on app type, the user may need to enter their credentials again or consent to permissions again.

As you said , when access token is expired, user is still log in on identity server. This is because the cookie in identity server hasn't expire yet. So you could do the following configuration to initiate the identity server to let the cookie expire quickly:

            var builder = services.AddIdentityServer(options =>
            {
                options.EmitStaticAudienceClaim = true;
                options.Authentication.CookieLifetime = TimeSpan.FromSeconds(5);
                options.Authentication.CookieSlidingExpiration = false;
            })             
                .AddDeveloperSigningCredential()
                .AddInMemoryIdentityResources(Config.IdentityResources)
                .AddInMemoryApiScopes(Config.ApiScopes)
                .AddInMemoryClients(Config.Clients);

Then you will see you will be redirect to login page when you access token expired. So you are "offline" in this circumstance.