OpenID Connect: inconsistent refresh token behaviour between different Identity Providers

I'm implementing a Service Provider and currently observing inconsistent behaviour by different Identity Providers, regarding getting refresh tokens. I'm going to attach my Service Provider golang code in the bottom, in case it might help someone or clarify my question.

I'm doing the authorization_code flow, by redirecting a login request to */authn endpoint with query parameter access_type=offline. Then, the second step is receiving the authorization code on the callback endpoint, then calling the */token endpoint to exchange the code for access and refresh tokens.

I've tried this flow with 3 different Identity Providers and found the following results:

1. OneLogin (https://openid-connect.onelogin.com/oidc): it was enough to add the query parameter access_type=offline to receive refresh tokens.
2. Okta (https://my-company.okta.com): adding access_type=offline was not enough. I needed to add offline_access to Scopes parameter of the request in the first step (authn). This configuration also worked for OneLogin!

3. Google (https://accounts.google.com): With Google however, the scope offline_access is not supported and 400 BAD REQUEST is returned:

   Some requested scopes were invalid. {valid=[openid, https://www.googleapis.com/auth/userinfo.profile, https://www.googleapis.com/auth/userinfo.email], invalid=[offline_access]}

   The only thing that worked with Google was removing offline_access from Scopes and adding the query parameter prompt with the value consent. This however, doesn't work with Okta or OneLogin...

Am I missing something, or should I provide a custom authorization flow implementation to every IdP, in order to support refresh tokens?

Seems pretty strange, considering that the protocol is fully specced out.

package openidconnect

import (
    "context"
    "encoding/json"
    "net/http"
    "os"

    oidc "github.com/coreos/go-oidc"
    "golang.org/x/oauth2"
)
var oidcClientID = getEnv("****", "OIDC_CLIENT_ID")
var oidcClientSecret = getEnv("****", "OIDC_CLIENT_SECRET")
var oidcProvider = getEnv("****", "OIDC_PROVIDER")

var oidcLoginURI = "/v1/oidc_login"
var oidcCallbackURI = "/v1/oidc_callback"
var hostname = getEnv("http://localhost:8080", "HOSTNAME")

func getEnv(defaultValue, key string) string {
    val := os.Getenv(key)
    if val == "" {
        return defaultValue
    }
    return val
}

//InitOpenIDConnect initiates open ID connect SSO
func InitOpenIDConnect() error {
    ctx := context.Background()

    provider, err := oidc.NewProvider(ctx, oidcProvider)
    if err != nil {
        return err
    }

    // Configure an OpenID Connect aware OAuth2 client.
    oidcConfig := oauth2.Config{
        ClientID:     oidcClientID,
        ClientSecret: oidcClientSecret,
        RedirectURL:  hostname + oidcCallbackURI,

        // Discovery returns the OAuth2 endpoints.
        Endpoint: provider.Endpoint(),

        // "openid" is a required scope for OpenID Connect flows.

        Scopes: []string{oidc.ScopeOpenID, "profile", "email"},
        // TODO: For Okta and OneLogin, add oidc.ScopeOfflineAccess Scope for refresh token.
        // Removed for now because Google API returns 400 when it is set.
    }

    handleOIDCLogin(&oidcConfig)
    handleOIDCCallback(provider, &oidcConfig)

    return nil
}

var approvalPromptOption = oauth2.SetAuthURLParam("prompt", "consent")

func handleOIDCLogin(config *oauth2.Config) {
    state := "foobar" // Don't do this in production.

    http.HandleFunc(oidcLoginURI, func(w http.ResponseWriter, r *http.Request) {
        // approval prompt option is required for getting refresh token from Google API
        redirectURL := config.AuthCodeURL(state, oauth2.AccessTypeOffline, approvalPromptOption)
        http.Redirect(w, r, redirectURL, http.StatusFound)
    })
}

func handleOIDCCallback(provider *oidc.Provider, config *oauth2.Config) {
    state := "foobar" // Don't do this in production.
    ctx := context.Background()

    http.HandleFunc(oidcCallbackURI, func(w http.ResponseWriter, r *http.Request) {
        if r.URL.Query().Get("state") != state {
            http.Error(w, "state did not match", http.StatusBadRequest)
            return
        }

        code := r.URL.Query().Get("code")

        oauth2Token, err := config.Exchange(ctx, code)
        if err != nil {
            http.Error(w, "Failed to exchange token: "+err.Error(), http.StatusInternalServerError)
            return
        }

        tokenSource := config.TokenSource(ctx, oauth2Token)
        refreshedToken, err := tokenSource.Token()
        if err != nil {
            http.Error(w, "Failed to get refresh token: "+err.Error(), http.StatusInternalServerError)
            return
        }

        userInfo, err := provider.UserInfo(ctx, oauth2.StaticTokenSource(oauth2Token))
        if err != nil {
            http.Error(w, "Failed to get userinfo: "+err.Error(), http.StatusInternalServerError)
            return
        }

        resp := struct {
            OAuth2Token *oauth2.Token
            UserInfo    *oidc.UserInfo
        }{oauth2Token, userInfo}
        data, err := json.MarshalIndent(resp, "", "    ")
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
        w.Write(data)
    })
}

I unfortunately think that the various providers implement this part differently. Okta seems to be the most compliant of these (requiring offline_access as scope is what the OIDC specification describes.

Making the scope values configurable, and possibly also making it possible to configure custom parameters (such as the access_type parameter) would be a way to avoid completely custom implementations for each provider.

The prompt parameter is part of the specification so making that configurable might be a good idea anyway.