I'm running Superset inside a docker container. installed it thru the recommended method explained on the Superset site (clone the GIT project), then tailored the docker-compose-non-dev.yml, .env-non-dev and requirements-local.txt and superset_config.py and pulled the images... it starts and i can log into it. HOWEVER. It keeps saying that it found a default secret key
superset_worker_beat | --------------------------------------------------------------------------------
superset_worker_beat | WARNING
superset_worker_beat | --------------------------------------------------------------------------------
superset_worker_beat | A Default SECRET_KEY was detected, please use superset_config.py to override it.
superset_worker_beat | Use a strong complex alphanumeric string and use a tool to help you generate
superset_worker_beat | a sufficiently random sequence, ex: openssl rand -base64 42
superset_worker_beat | --------------------------------------------------------------------------------
superset_worker_beat | --------------------------------------------------------------------------------
It keeps saying that it will use default cache setting and they should be redefined properly to avoid data loss:
# superset_worker_beat | Falling back to the built-in cache, that stores data in the metadata database, for the following cache: `FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`, `MemcachedCache` or another dedicated caching backend for production deployments
superset_worker_beat | 2023-07-28 12:47:06,744:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: FILTER_STATE_CACHE_CONFIG. It is recommended to use RedisCache, MemcachedCache or another dedicated caching backend for production deployments
# superset_worker_beat | Falling back to the built-in cache, that stores data in the metadata database, for the following cache: EXPLORE_FORM_DATA_CACHE_CONFIG. It is recommended to use RedisCache, MemcachedCache or another dedicated caching backend for production deployments
# superset_worker_beat | 2023-07-28 12:47:06,748:WARNING:superset.utils.cache_manager:Falling back to the built-in cache, that stores data in the metadata database, for the following cache: EXPLORE_FORM_DATA_CACHE_CONFIG. It is recommended to use RedisCache, MemcachedCache or another dedicated caching backend for production deployments
It authenticates to the default Superset page.
It keeps saying that it found a default secret key: I've defined both:
SECRET_KEY = "SuperSecretCodeIKnow"
SUPERSET_SECRET_KEY = "SuperSecretCodeIKnow"
It keeps saying that it will use default cache setting and they should be redefined properly to avoid data loss: I've defined all the proper blocks acording to the SS documentation on the superset_config.py file:
REDIS_HOST = get_env_variable("REDIS_HOST")
REDIS_PORT = get_env_variable("REDIS_PORT")
REDIS_CELERY_DB = get_env_variable("REDIS_CELERY_DB", "0")
REDIS_RESULTS_DB = get_env_variable("REDIS_RESULTS_DB", "1")
RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")
CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_DEFAULT_TIMEOUT": 300,
"CACHE_KEY_PREFIX": "superset_",
"CACHE_REDIS_HOST": REDIS_HOST,
"CACHE_REDIS_PORT": REDIS_PORT,
"CACHE_REDIS_DB": REDIS_RESULTS_DB,
}
DATA_CACHE_CONFIG = CACHE_CONFIG
FILTER_STATE_CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 86400,
"CACHE_REDIS_HOST": REDIS_HOST,
"CACHE_REDIS_PORT": REDIS_PORT,
"CACHE_REDIS_DB": 2,
'CACHE_KEY_PREFIX': 'superset_filter_',
EXPLORE_FORM_DATA_CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 86400,
'CACHE_KEY_PREFIX': 'superset_explore_form_',
"CACHE_REDIS_HOST": REDIS_HOST,
"CACHE_REDIS_PORT": REDIS_PORT,
"CACHE_REDIS_DB": 3,
'CACHE_REDIS_URL': 'redis://redis:6379/3'
}
It authenticates to the default Superset page.
Superset is running behind an NginX reverse proxy for the HTTPS capabilities (this works FINE).
I've added the proxy_fix settings to the superset_config.py:
ENABLE_PROXY_FIX = True
PROXY_FIX_CONFIG = {"x_for": 1, "x_proto": 1, "x_host": 1, "x_port": 0, "x_prefix": 1}
On this i'm trying to integrate SS with keycloak. i've tried the few recipes here on StackOverFlow, as well as a few other sites. Including the new flask.appbuilder capabilities for keycloak connections for authentications, by all accounts, it should be working, but all i get is always the SS login page.
my requirements-local.txt has the following packages:
itsdangerous==2.0.1
flask-oidc==1.4.0
Flask-OpenID==1.3.0
flask-appbuilder==4.2.0
my client_secret.json is like this one (with the correct URLs, Realms, ID,secret):
{
"web": {
"issuer": "https://keycloak.local/realms/master",
"auth_uri": "https://keycloak.local/realms/master/protocol/openid-connect/auth",
"client_id": "myclientid",
"client_secret": "4H6sra49h4BpV4J1SIGlx0YedwqSUNVx",
"redirect_uris": ["https://superset.local/*"],
"userinfo_uri": "http://keycloak:8080/realms/master/protocol/openid-connect/userinfo",
"token_uri": "http://keycloak:8080/realms/master/protocol/openid-connect/token",
"token_introspection_uri": "http://keycloak:8080/realms/master/protocol/openid-connect/token/introspect"
}
}
I've added the following code to superset_config.py:
'''
---------------------------KEYCLOACK ----------------------------
'''
curr = os.path.abspath(os.getcwd())
AUTH_TYPE = AUTH_OID
OIDC_CLIENT_SECRETS = curr + '/docker/pythonpath_dev/client_secret.json'
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
OIDC_OPENID_REALM: 'master'
OIDC_INTROSPECTION_AUTH_METHOD: 'client_secret_post'
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'
OIDC_VALID_ISSUERS = ['https://[url]/realms/[realm]']
'''
--------------------------------------------------------------
'''
and created a file named keycloak_security_manager.py and placed it in the /pythonpath_dev path with the following:
from flask import redirect, request
from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging
import urllib.parse
class OIDCSecurityManager(SupersetSecurityManager):
def __init__(self, appbuilder):
super(OIDCSecurityManager, self).__init__(appbuilder)
if self.auth_type == AUTH_OID:
self.oid = OpenIDConnect(self.appbuilder.get_app)
self.authoidview = AuthOIDCView
class AuthOIDCView(AuthOIDView):
@expose('/login/', methods=['GET', 'POST'])
def login(self, flag=True):
sm = self.appbuilder.sm
oidc = sm.oid
@self.appbuilder.sm.oid.require_login
def handle_login():
user = sm.auth_user_oid(oidc.user_getfield('email'))
if user is None:
info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'),
info.get('email'), sm.find_role('Gamma'))
login_user(user, remember=False)
return redirect(self.appbuilder.get_url_for_index)
return handle_login()
@expose('/logout/', methods=['GET', 'POST'])
def logout(self):
oidc = self.appbuilder.sm.oid
oidc.logout()
super(AuthOIDCView, self).logout()
redirect_url = urllib.parse.quote_plus(request.url_root.strip('/') + self.appbuilder.get_url_for_login)
return redirect(
oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?client_id='+ oidc.client_secrets.get('client_id')+'&post_logout_redirect_uri=' + quote(redirect_url))
On the keycloak side, i've created the realm, created the client, the roles inside the client, a user to whom i've assigned roles with the client but nothing... it ALWAYS pops the Superset auth page. I can login and use it but not thru keycloak.
I ended up starting afresh (benefits of using VMs)... I WAS using the default recommended paths and this last time went for SS 2.1.0. managed to make the SECRET_KEY, and CACHEs work, even got SS passing authentication to KC. (however now i get an error 500 after user validation). Also, was getting some gunicorn errors which were solved by adding gunincorn to the requirements-local.txt file so it gets upgraded upon start of the docker compose project.
defined the SECRET_KEY in my .env-non-dev file in the /docker dir (created with the 'openssql rand -base64 42' command)
and used the following as cache definitions inside my superset_config.py:
FILTER_STATE_CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 86400,
'CACHE_KEY_PREFIX': 'superset_filter_',
'CACHE_REDIS_URL': 'redis://localhost:6379/2'
}
EXPLORE_FORM_DATA_CACHE_CONFIG = {
'CACHE_TYPE': 'RedisCache',
'CACHE_DEFAULT_TIMEOUT': 86400,
'CACHE_KEY_PREFIX': 'superset_data_explorer_',
'CACHE_REDIS_URL': 'redis://localhost:6379/3'
}
CONTENT_SECURITY_POLICY_WARNING = False