[SOLVED] Issue with CDKTF Deployment of Azure Infrastructure in Azure DevOps Pipeline

Issue with CDKTF Deployment of Azure Infrastructure in Azure DevOps Pipeline

I'm encountering an issue while trying to deploy Azure infrastructure using CDKTF (azurerm provider) within an Azure DevOps pipeline. Here's a simplified version of the code I'm using:

//code setup...

class MyStack extends TerraformStack {
  constructor(scope: Construct, id: string, env: Environment) {
    super(scope, id);

    // Azure provider setup and backend configuration...

    const resGroup = new ResourceGroup(this, env.CreateId(resourceGroupId), {
      // Resource group configuration...
    });

    const mainStorageAccount = new StorageAccount(this, env.CreateId(storageAccountId), {
      // Storage account configuration...
    });

    const cdnProfile = new CdnProfile(this, env.CreateId(profileNameId), {
      // CDN profile configuration...
    });

    const cdnEndpointConfig = {
      // CDN endpoint configuration...
    };

    new CdnEndpoint(this, cdnEndpointConfig.name, cdnEndpointConfig);
  }
}

// App setup and environment variables...

const app = new App();

const tfEnvironmentType = process.env.TF_ENVIRONMENT || 'test';
const market = process.env.MARKET || 'A';
let env: Environment | undefined;

if (tfEnvironmentType && market) {
  env = new Environment(tfEnvironmentType, market);
} else {
  console.error('One or more environment variables are missing.');
  process.exit(1); // Exit the process with a non-zero status code to indicate an error
}

if (env) {
  const environmentID = env.GetID();
  new MyStack(app, 'src', env);
  app.synth();
}

Azure DevOps pipeline task for deploying with Market A:

- task: AzureCLI@2
  displayName: Running Deployment with Market XX
  inputs:
    azureSubscription: $(AZURE_SUBSCRIPTION)
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    addSpnToEnvironment: true
    failOnStandardError: true
    inlineScript: |
      cd infrastructure
      export ARM_CLIENT_ID=$servicePrincipalId
      export ARM_CLIENT_SECRET=$servicePrincipalKey
      export ARM_TENANT_ID=$tenantId
      export ARM_SUBSCRIPTION_ID=$AZURE_SUBSCRIPTION_ID
      export MARKET=A # Set Market variable to 'A'
      echo "Deploying infrastructure with Market: $MARKET"
      npm run deploy

Azure DevOps pipeline task for deploying with Market B:

- task: AzureCLI@2
  displayName: Running Deployment with Market DK
  inputs:
    azureSubscription: $(AZURE_SUBSCRIPTION)
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    addSpnToEnvironment: true
    failOnStandardError: true
    inlineScript: |
      cd infrastructure
      export ARM_CLIENT_ID=$servicePrincipalId
      export ARM_CLIENT_SECRET=$servicePrincipalKey
      export ARM_TENANT_ID=$tenantId
      export ARM_SUBSCRIPTION_ID=$AZURE_SUBSCRIPTION_ID
      export MARKET=B # Set Market variable to 'B'
      echo "Deploying infrastructure with Market: $MARKET"
      npm run deploy

package.json

    "scripts": {
      "deploy": "cdktf deploy $npm_config_TF_ENVIRONMENT --auto-approve",
      // Other scripts...
    }

I've noticed that deploying the infrastructure for Market B is causing the resources created for Market A to be destroyed. How can I deploy resources for both Market A and B without interfering with each other? Any help in successfully deploying Azure infrastructure using CDKTF within an Azure DevOps pipeline for different markets would be greatly appreciated.

Update:

Thanks @Martin, Besides using separate tfstate files, I found a fix by using different output parameters in my deployment:

Here's how I set it up:

"deploy:A": "MARKET=A cdktf deploy $npm_config_TF_ENVIRONMENT --auto-approve --output=cdktf.out.A",
"deploy:B": "MARKET=B cdktf deploy $npm_config_TF_ENVIRONMENT --auto-approve --output=cdktf.out.B"

So, with these commands:

deploy:A for the "A" market. deploy:B for the "B" market. This trick allowed me to customize the deployment process as needed. Just sharing this in case it helps someone else out! Cheers,

Solution

I only have experience with AWS cdktf but I think you are missing defining separated state files like for each stack. like:

new aws.provider.AwsProvider(this, 'AWS', {
  region: env.region,
  defaultTags: [{ tags }],
});

new cdktf.S3Backend(this, {
  bucket: `martti-data-management-${env.name}-tf-state-${env.region}`,
  key: `accounts/${env.name}/bootstrap/${name}-terraform.tfstate`,
  region: env.region,
});