I want to automate express route circuit & Express Route Gateway for Equinix provider using Terraform and deploy it through azure pipeline

I want to create a module related to express route circuit & Express Route Gateway for Equinix provider using Terraform.I want to run this module to be deployed through pipeline as a separate action only after executing all the resources deployed. Pipeline snippet

In the image, i should have a option to choose to the creation of express route whenever required instead of deploying it along with other resources that this pipeline executes when i choose plan-deploy option. And if it is possible how does the state file works. will there be any dependencies from previous deployed resources through plan-deploy option

Providing my pipeline.yml file

trigger: none
parameters:
  - name: env
    type: string
    default: dev
    values:
      - dev
      - prod

  - name: regions
    type: string
    default: westeurope
    values:
      - westeurope
  
  - name: action
    type: string
    default: deploy
    values:
  
      - plan
      - plan-deploy
      - plan-destroy
      - Initialize
      - marketplace-subscribe
    #Here I need an action , when i choose this action , I should be able to initialize,plan and apply for expressroute resources(only expressroute module should be executed)

variables:
  - template: variables/${{ parameters.env }}/default.yml
  - template: variables/${{ parameters.env }}/${{ parameters.regions }}/default.yml
  - name: additionalTerraformStatements
    ${{ if eq( parameters['action'], 'plan-destroy') }}:
      value: "-destroy"
  
# Stage: Azure Marketplace subscription terms    
stages:

## Initialize the new setup for any new landing zone
  - stage: Initialize
    condition: eq('${{ parameters.action }}', 'Initialize')
    displayName: 'Terraform Initialize'
    jobs:
    - template: templates/Initialize.yml
      parameters:
        hubSubscription: '$(hubSubscription)'
        agentPool: '$(agentPool)'
        agentImage: '$(agentImage)'
        updateVariablesScript: '$(updateVariablesScript)'
        marketplaceScript: '$(marketplaceScript)'
        parametersFile: '$(parametersFile)'
        parametersPath: '$(parametersPath)/${{ parameters.env }}/${{ parameters.regions }}'
        terraformDirectory: '$(terraformDirectory)'
        marketplacePublisher: '$(marketplacePublisher)'
        marketplaceProduct: '$(marketplaceProduct)'
        location: '$(locatione)'
        fw_size: '$(fw_size)'
        fw_bundle: '$(fw_bundle)'
        panos_version: '$(panos_version)'
        adminuser: '$(adminuser)'
        subscription_id: '$(subscription_id)'
        prefix: '$(prefix)'
        resourceGroupPaloAlto: '$(resourceGroupPaloAlto)'
        resourceGroupStorage: '$(resourceGroupStorage)'
        storageAccount: '$(storageAccount)'
        StorageAccountFileShare: '$(StorageAccountFileShare)'
        StorageAccountFileShareDirectory: '$(StorageAccountFileShareDirectory)'
        blobContainerTerraform: '$(blobContainerTerraform)'
        tfStateFile: '$(tfStateFile)'
        tfplan: '$(tfplan)'
  
  - stage: plan
    condition: or(eq('${{ parameters.action }}', 'plan'),eq('${{ parameters.action }}', 'plan-deploy'),eq('${{ parameters.action }}', 'plan-destroy'))
    displayName: 'Terraform Plan'
    jobs:
    - template: templates/AnalyzeChanges.yml
      parameters:
        hubSubscription: '$(hubSubscription)'
        agentPool: '$(agentPool)'
        agentImage: '$(agentImage)'
        updateVariablesScript: '$(updateVariablesScript)'
        marketplaceScript: '$(marketplaceScript)'
        parametersFile: '$(parametersFile)'
        parametersPath: '$(parametersPath)/${{ parameters.env }}/${{ parameters.regions }}'
        terraformDirectory: '$(terraformDirectory)'
        storageAccountKey: '$(storageAccountKey)'
        localAdminSecret: '$(localAdminSecret)'
        artifacts: '$(artifacts)'
        additionalTerraformStatements: '${{variables.additionalTerraformStatements}}'
        terraformvar: '$(terraform-var)'
        location: '$(locatione)'
        fw_size: '$(fw_size)'
        fw_bundle: '$(fw_bundle)'
        panos_version: '$(panos_version)'
        marketplacePublisher: '$(marketplacePublisher)'
        marketplaceProduct: '$(marketplaceProduct)'
        adminuser: '$(adminuser)'
        subscription_id: '$(subscription_id)'
        prefix: '$(prefix)'
        resourceGroupPaloAlto: '$(resourceGroupPaloAlto)'
        resourceGroupStorage: '$(resourceGroupStorage)'
        storageAccount: '$(storageAccount)'
        StorageAccountFileShare: '$(StorageAccountFileShare)'
        StorageAccountFileShareDirectory: '$(StorageAccountFileShareDirectory)'
        blobContainerTerraform: '$(blobContainerTerraform)'
        tfStateFile: '$(tfStateFile)'
        tfplan: '$(tfplan)'
  
  - stage: apply
    condition: or(eq('${{ parameters.action }}', 'deploy'),eq('${{ parameters.action }}', 'plan-deploy'),eq('${{ parameters.action }}', 'plan-destroy'))
    displayName: 'Terraform Apply'
    jobs:
    - template: templates/ApplyChanges.yml
      parameters:
        hubSubscription: '$(hubSubscription)'
        agentPool: '$(agentPool)'
        agentImage: '$(agentImage)'
        updateVariablesScript: '$(updateVariablesScript)'
        parametersFile: '$(parametersFile)'
        storageAccountKey: '$(storageAccountKey)'
        artifacts: '$(artifacts)'
        additionalTerraformStatements: '${{variables.additionalTerraformStatements}}'
        location: '$(locatione)'
        fw_size: '$(fw_size)'
        fw_bundle: '$(fw_bundle)'
        panos_version: '$(panos_version)'
        marketplacePublisher: '$(marketplacePublisher)'
        marketplaceProduct: '$(marketplaceProduct)'
        adminuser: '$(adminuser)'
        subscription_id: '$(subscription_id)'
        prefix: '$(prefix)'
        resourceGroupPaloAlto: '$(resourceGroupPaloAlto)'
        resourceGroupStorage: '$(resourceGroupStorage)'
        storageAccount: '$(storageAccount)'
        StorageAccountFileShare: '$(StorageAccountFileShare)'
        StorageAccountFileShareDirectory: '$(StorageAccountFileShareDirectory)'
        blobContainerTerraform: '$(blobContainerTerraform)'
        tfStateFile: '$(tfStateFile)'
        tfplan: '$(tfplan)'

All the scripts are added in templates folder, like analyzechanges.yml, applychanges.yml

I need a suggestion here, because i am concerned about my state file. I can create a separate directory for express route to deploy the resource , if i need to destroy it, then i need a way to perform all the terraform actions plan,apply,deploy,destroy

Automate express route circuit & Express Route Gateway for Equinix provider using Terraform and deploy it through azure pipeline

In order to manage dependencies and lifecyle independently you can use separate state files for different parts of your infrastructure.

Try to include the expressroute-plan, expressroute-deploy, expressroute-destroy in action specifically for the ExpressRoute resources.

yaml:

- name: action
  type: string
  default: deploy
  values:
    - plan
    - plan-deploy
    - plan-destroy
    - Initialize
    - marketplace-subscribe
    - expressroute-plan
    - expressroute-deploy
    - expressroute-destroy

now added an additional stage for the ExpressRoute

- stage: expressroute
  condition: eq('${{ parameters.action }}', 'expressroute-action')
  displayName: 'Terraform ExpressRoute'
  jobs:
    - template: templates/ExpressRoute.yml
      parameters:
        hubSubscription: '$(hubSubscription)'
        agentPool: '$(agentPool)'
        agentImage: '$(agentImage)'
        terraformDirectory: '$(expressrouteTerraformDirectory)' # Separate directory for ExpressRoute
        tfStateFile: '$(expressrouteTfStateFile)' # Separate state file for ExpressRoute

If you feel that there realtions between the deployments use this below to access the information from the existing deployment.

data "terraform_remote_state" "core_infra" {
  backend = "azurerm"
  config = {
    storage_account_name = "coreinfraaccount"
    container_name       = "core"
    key                  = "coreinfra.tfstate"
  }
}

For more information on how to split state file

refer:

https://support.hashicorp.com/hc/en-us/articles/7955227415059-How-to-Split-State-Files?utm_

https://thomasthornton.cloud/2021/08/04/conditional-variables-in-azure-devops-pipelines/