python django google-app-engine django-settings django-deployment

Error in deploying Django app on Google cloud using appengine "django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable"

I am trying to deploy my own django app on google cloud. I'm following this documentation by Google Cloud to deploy the app. I have changed the settings.py file of my app according to the settings.py file of the sample app provided by Google.

I have created a Django environment file as a Secret Manager secret by following the same documentation. its format is

echo DATABASE_URL=postgres://DATABASE_NAME:DATABASE_USER_PASSWORD@//cloudsql/PROJECT_ID:REGION:INSTANCE_ID/CRRathod > .env
echo GS_BUCKET_NAME=my-portfolio-361305_crrathod-bucket >> .env
echo SECRET_KEY=$(cat/dev/urandom | LC_ALL=C tr -dc '[:alpha:]'| fold -w 50 | head -n1) >> .env

After entering next command as documentation python manage.py makemigrations I'm getting errors as

(env) C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog>python manage.py makemigrations
Invalid line: echo DATABASE_URL=postgres://DATABASE_NAME:DATABASE_USER_PASSWORD@//cloudsql/PROJECT_ID:REGION:INSTANCE_ID/CRRathod > .env
Invalid line: echo GS_BUCKET_NAME=my-portfolio-361305_crrathod-bucket >> .env
Invalid line: echo SECRET_KEY=$(cat/dev/urandom | LC_ALL=C tr -dc '[:alpha:]'| fold -w 50 | head -n1) >> .env
Invalid line: echo DATABASE_URL=postgres://CRRathod:CRroot@123@//cloudsql/my-portfolio-361305:asia-south1:crrathod/CRRathod > .env
Invalid line: echo GS_BUCKET_NAME=my-portfolio-361305_crrathod-bucket >> .env
Invalid line: echo SECRET_KEY=$(cat/dev/urandom | LC_ALL=C tr -dc '[:alpha:]'| fold -w 50 | head -n1) >> .env
Invalid line: echo DATABASE_URL=postgres://CRRathod:CRroot@123@//cloudsql/my-portfolio-361305:asia-south1:crrathod/CRRathod > .env
Invalid line: echo GS_BUCKET_NAME=my-portfolio-361305_crrathod-bucket >> .env
Invalid line: echo SECRET_KEY=$(cat/dev/urandom | LC_ALL=C tr -dc '[:alpha:]'| fold -w 50 | head -n1) >> .env
Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\environ\environ.py", line 403, in get_value
    value = self.ENVIRON[var_name]
  File "C:\Users\LONAR\AppData\Local\Programs\Python\Python38\lib\os.py", line 675, in __getitem__
    raise KeyError(key) from None
KeyError: 'SECRET_KEY'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    main()
  File "manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\management\__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\management\__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\management\base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\management\base.py", line 455, in execute
    self.check()
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\management\base.py", line 487, in check
    all_issues = checks.run_checks(
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\checks\registry.py", line 88, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\core\checks\caches.py", line 64, in check_file_based_cache_is_absolute
    for alias, config in settings.CACHES.items():
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\conf\__init__.py", line 87, in __getattr__
    self._setup(name)
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\conf\__init__.py", line 74, in _setup
    self._wrapped = Settings(settings_module)
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\django\conf\__init__.py", line 183, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "C:\Users\LONAR\AppData\Local\Programs\Python\Python38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\crrathod\settings.py", line 59, in <module>
    SECRET_KEY = env("SECRET_KEY")
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\environ\environ.py", line 197, in __call__
    return self.get_value(
  File "C:\Program Files (x86)\Google\Cloud SDK\Portfolio-blog\env\lib\site-packages\environ\environ.py", line 407, in get_value
    raise ImproperlyConfigured(error_msg) from exc
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

I'm hiding some data here as security precaution and representing its pattern only.

settings.py file

from pathlib import Path
import os
import io
from urllib.parse import urlparse

import environ
from google.cloud import secretmanager

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# [START gaestd_py_django_secret_config]
env = environ.Env(DEBUG=(bool, False))
env_file = os.path.join(BASE_DIR, ".env")

if os.path.isfile(env_file):
    # Use a local secret file, if provided

    env.read_env(env_file)
# [START_EXCLUDE]
elif os.getenv("TRAMPOLINE_CI", None):
    # Create local settings if running with CI, for unit testing

    placeholder = (
        f"SECRET_KEY=a\n"
        f"DATABASE_URL=sqlite://{os.path.join(BASE_DIR, 'db.sqlite3')}"
    )
    env.read_env(io.StringIO(placeholder))
# [END_EXCLUDE]
elif os.environ.get("GOOGLE_CLOUD_PROJECT", None):
    # Pull secrets from Secret Manager
    project_id = os.environ.get("GOOGLE_CLOUD_PROJECT")

    client = secretmanager.SecretManagerServiceClient()
    settings_name = os.environ.get("SETTINGS_NAME", "django_settings")
    name = f"projects/{project_id}/secrets/{settings_name}/versions/latest"
    payload = client.access_secret_version(name=name).payload.data.decode("UTF-8")

    env.read_env(io.StringIO(payload))
else:
    raise Exception("No local .env or GOOGLE_CLOUD_PROJECT detected. No secrets found.")
# [END gaestd_py_django_secret_config]



SECRET_KEY = env("SECRET_KEY")


# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = env("DEBUG")

# [START gaestd_py_django_csrf]
# SECURITY WARNING: It's recommended that you use this when
# running in production. The URL will be known once you first deploy
# to App Engine. This code takes the URL and converts it to both these settings formats.
APPENGINE_URL = env("APPENGINE_URL", default=None)
if APPENGINE_URL:
    # Ensure a scheme is present in the URL before it's processed.
    if not urlparse(APPENGINE_URL).scheme:
        APPENGINE_URL = f"https://{APPENGINE_URL}"

    ALLOWED_HOSTS = [urlparse(APPENGINE_URL).netloc]
    CSRF_TRUSTED_ORIGINS = [APPENGINE_URL]
    SECURE_SSL_REDIRECT = True
else:
    ALLOWED_HOSTS = ["*"]
# [END gaestd_py_django_csrf]


# Application definition

INSTALLED_APPS = [
    'portpolio.apps.PortpolioConfig',
    'CRblogs.apps.CrblogsConfig',
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django_social_share',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'crrathod.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS':  [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'crrathod.wsgi.application'


# Database
# [START db_setup]
# [START gaestd_py_django_database_config]
# Use django-environ to parse the connection string
DATABASES = {"default": env.db()}

# If the flag as been set, configure to use proxy
if os.getenv("USE_CLOUD_SQL_AUTH_PROXY", None):
    DATABASES["default"]["HOST"] = "127.0.0.1"
    DATABASES["default"]["PORT"] = 8088

# [END gaestd_py_django_database_config]
# [END db_setup]

# Use a in-memory sqlite3 database when testing in CI systems
# TODO(glasnt) CHECK IF THIS IS REQUIRED because we're setting a val above
if os.getenv("TRAMPOLINE_CI", None):
    DATABASES = {
        "default": {
            "ENGINE": "django.db.backends.sqlite3",
            "NAME": os.path.join(BASE_DIR, "db.sqlite3"),
        }
    }



# Password validation
# https://docs.djangoproject.com/en/4.0/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.0/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.0/howto/static-files/

# STATICFILES_DIRS =[
#     os.path.join(BASE_DIR, 'static')
# ]

# STATIC_ROOT = os.path.join(BASE_DIR,'assets')

STATIC_URL = 'static/'
MEDIA_URL = '/media/'

if DEBUG:
    STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static')]
else:
    STATIC_ROOT = os.path.join(BASE_DIR, 'static')

MEDIA_ROOT = os.path.join(BASE_DIR,'media')

# Default primary key field type
# https://docs.djangoproject.com/en/4.0/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

I'm not able to solve the error after lots of attempts. please help me thank you

Solution

I have solved this issue by removing some extra spaces from my secret manager file i.e. .env file as I mentioned in the question previously my secret manager code was

echo DATABASE_URL=postgres://DATABASE_USERNAME:DATABASE_USER_PASSWORD@//cloudsql/PROJECT_ID:REGION:INSTANCE_ID/CRRathod > .env
echo GS_BUCKET_NAME=my-portfolio-361305_crrathod-bucket >> .env
echo SECRET_KEY=$(cat/dev/urandom | LC_ALL=C tr -dc '[:alpha:]'| fold -w 50 | head -n1) >> .env

I had changed it to

echo DATABASE_URL=postgres://DATABASE_USERNAME:DATABASE_USER_PASSWORD@//cloudsql/PROJECT_ID:REGION:INSTANCE_ID/CRRathod>.env
echo GS_BUCKET_NAME=my-portfolio-361305_crrathod-bucket>>.env
echo SECRET_KEY=$(cat/dev/urandom | LC_ALL=C tr -dc '[:alpha:]'| fold -w 50 | head -n1)>>.env

Just removed all extra spaces

Then set the secret key value from the terminal as set SECRET_KEY=my_secret_key_value