I am working on upgrading templates from terraform 0.12.31 to 0.13.7, we need to ensure that we have an automatic system for dealing with deployments that were created under the older version.
An issue I am working through is that I removed the use of all null providers in the move. When I attempt to apply or plan on a state file created on 0.12 when using terraform version 0.13 I recieve the following error:
$ terraform plan --var-file MY_VAR_FILE.json
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
Error: Provider configuration not present
To work with
module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost
its original provider configuration at
provider["registry.terraform.io/-/null"] is required, but it has been removed.
This occurs when a provider configuration is removed while objects created by
that provider still exist in the state. Re-add the provider configuration to
destroy
module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost,
after which you can remove the provider configuration again.
Error: Provider configuration not present
To work with
module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master
its original provider configuration at
provider["registry.terraform.io/-/null"] is required, but it has been removed.
This occurs when a provider configuration is removed while objects created by
that provider still exist in the state. Re-add the provider configuration to
destroy
module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master,
after which you can remove the provider configuration again.
Error: Provider configuration not present
To work with
module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config its
original provider configuration at provider["registry.terraform.io/-/null"] is
required, but it has been removed. This occurs when a provider configuration
is removed while objects created by that provider still exist in the state.
Re-add the provider configuration to destroy
module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config, after
which you can remove the provider configuration again.
My manual solution is to run terraform state rm on all the modules listed:
terraform state rm module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config
terraform state rm module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost
terraform state rm module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master
I would like to know how to do this automatically to enable a script to make these changes.
Is there some kind of terraform command I can use to list out these removed modules without the extra test so I can loop through runs of terraform state rm to remove them from the state file?
Or is there some kind of terraform command that can automatically do this in a generic manner like terraform state rm -all-not-present?
There's a few possibilities. Without the source code of the module it's difficult to say so providing that might be helpful.
A couple of suggestions
Remove the .terraform directory (normally in the directory you're running the init, plan, and apply from). The an older version of the module could be cached that still contains the null references.
Using Terraform Refresh you should be able to scan infra and bring state into alignment.
Can be dangerous, not recommended by Hashicorp.
The state rm command like you've suggested could help here and is fairly safe. You have an option of a --dry-run and you point to resources specifically
Using terraform state rm like you've suggested to manually remove those resources within the modules in state. Again, you want to check that the module reference isn't pointing to an old version of the module or caching an old one or they will just be recreated.
No there's not a rm --all-missing but if you know it's all the null data sources that will be missing you could use terraform state ls to list all those resources, then iterate over each of them removing them in a loop.