I'm having a hard time setting up Substitution Variables for Google Cloud Build. I keep getting an invalid value for 'build.substitutions' error

I'm relatively new to Google Cloud from a developer standpoint. I have setup a basic website and configured Google Cloud Build to run any time I push a change to GitHub. This was working well until I decided to integrate a Firebase DB. When the Cloud Build runs I get the following error.

Your build failed to run: generic::invalid_argument: invalid value for 'build.substitutions': key in the template "FIREBASE_API_KEY" is not a valid built-in substitution

Below is a copy of my current cloudbuild.yaml.

steps:
  - name: "gcr.io/cloud-builders/npm"
    dir: "dev"
    entrypoint: "bash"
    args:
      - "-c"
      - |
        set -e

        # Install dependencies
        npm install

  - name: "gcr.io/cloud-builders/npm"
    dir: "dev"
    entrypoint: "bash"
    env:
      - "FIREBASE_API_KEY=${_FIREBASE_API_KEY}"
      - "FIREBASE_AUTH_DOMAIN=${_FIREBASE_AUTH_DOMAIN}"
      - "FIREBASE_PROJECT_ID=${_FIREBASE_PROJECT_ID}"
      - "FIREBASE_STORAGE_BUCKET=${_FIREBASE_STORAGE_BUCKET}"
      - "FIREBASE_MESSAGING_SENDER_ID=${_FIREBASE_MESSAGING_SENDER_ID}"
      - "FIREBASE_APP_ID=${_FIREBASE_APP_ID}"
    args:
      - "-c"
      - |
        set -e

        # Create .env file with environment variables
        echo "FIREBASE_API_KEY=${FIREBASE_API_KEY}" > .env
        echo "FIREBASE_AUTH_DOMAIN=${FIREBASE_AUTH_DOMAIN}" >> .env
        echo "FIREBASE_PROJECT_ID=${FIREBASE_PROJECT_ID}" >> .env
        echo "FIREBASE_STORAGE_BUCKET=${FIREBASE_STORAGE_BUCKET}" >> .env
        echo "FIREBASE_MESSAGING_SENDER_ID=${FIREBASE_MESSAGING_SENDER_ID}" >> .env
        echo "FIREBASE_APP_ID=${FIREBASE_APP_ID}" >> .env

        # Source environment variables
        source .env

        # Build the project
        npm run build

  - name: "gcr.io/cloud-builders/gcloud"
    dir: "dev"
    entrypoint: "bash"
    args:
      - "-c"
      - |
        set -e

        # Source environment variables
        source .env

        # Deploy the app
        gcloud app deploy --project=my-project --appyaml=app.yaml

timeout: "1200s"
options:
  logging: CLOUD_LOGGING_ONLY

substitutions:
  _FIREBASE_API_KEY: ""
  _FIREBASE_AUTH_DOMAIN: ""
  _FIREBASE_PROJECT_ID: ""
  _FIREBASE_STORAGE_BUCKET: ""
  _FIREBASE_MESSAGING_SENDER_ID: ""
  _FIREBASE_APP_ID: ""

I've tried numeration iterations of modifying my cloudbuild.yaml file to get this to work. Originally, by DB Secrets were in Google Secret Manager and I was trying to pull from there, but I couldn't get that to work. Now I am setting the secrets as Substitution Variables within the Cloud Build Trigger.

The snippet below is where I'm using the variables. The .env file is still being used but I'm open to better/cleaner solutions.

import { initializeApp } from "firebase/app"
import { getFirestore, doc, getDoc, setDoc, updateDoc } from "firebase/firestore"

// Initialize Firebase with environment variables
const firebaseConfig = {
  apiKey: process.env.FIREBASE_API_KEY,
  authDomain: process.env.FIREBASE_AUTH_DOMAIN,
  projectId: process.env.FIREBASE_PROJECT_ID,
  storageBucket: process.env.FIREBASE_STORAGE_BUCKET,
  messagingSenderId: process.env.FIREBASE_MESSAGING_SENDER_ID,
  appId: process.env.FIREBASE_APP_ID
}

// Initialize Firebase
const app = initializeApp(firebaseConfig)
const db = getFirestore(app)

I'm not sure where things are going wrong. I tried using ChatGPT and that has complicated things even more (and been very unhelpful). Thanks in advance for any pointers!!!

10 years in IT/Development and this is my first StackExchange post!! woohoo!

No shame posting your first question after 10 years! Everything new is challenging, and you will see the answer pretty "weird"!

In your step, you are defining ENV VARS that take the value of the substitution variables. That part is OK

Then, in your step, you are calling your env var and it's not correct. With Cloud Build if you use a single $, you are calling the substitution variables. If your use the double $, this time you will call the env vars! Detail here in the doc

Try this instead

...
 - name: "gcr.io/cloud-builders/npm"
    dir: "dev"
    entrypoint: "bash"
    env:
      - "FIREBASE_API_KEY=${_FIREBASE_API_KEY}"
      - "FIREBASE_AUTH_DOMAIN=${_FIREBASE_AUTH_DOMAIN}"
      - "FIREBASE_PROJECT_ID=${_FIREBASE_PROJECT_ID}"
      - "FIREBASE_STORAGE_BUCKET=${_FIREBASE_STORAGE_BUCKET}"
      - "FIREBASE_MESSAGING_SENDER_ID=${_FIREBASE_MESSAGING_SENDER_ID}"
      - "FIREBASE_APP_ID=${_FIREBASE_APP_ID}"
    args:
      - "-c"
      - |
        set -e

        # Create .env file with environment variables
        echo "FIREBASE_API_KEY=$${FIREBASE_API_KEY}" > .env
        echo "FIREBASE_AUTH_DOMAIN=$${FIREBASE_AUTH_DOMAIN}" >> .env
        echo "FIREBASE_PROJECT_ID=$${FIREBASE_PROJECT_ID}" >> .env
        echo "FIREBASE_STORAGE_BUCKET=$${FIREBASE_STORAGE_BUCKET}" >> .env
        echo "FIREBASE_MESSAGING_SENDER_ID=$${FIREBASE_MESSAGING_SENDER_ID}" >> .env
        echo "FIREBASE_APP_ID=$${FIREBASE_APP_ID}" >> .env

        # Source environment variables
        source .env

        # Build the project
        npm run build
...

That being said, what's the purpose to create a .env file? Just to source it just after? The env var already exist in this step.

For the AppEngine deploy step, the source before the gcloud app deploy is useless; the command does not take into account the env vars. Your .env file is only useful if the app engine deployment need it.