Handle reordered list attributes in Terraform Provider

I am implementing a Terraform provider using the Terraform Plugin Framewok.

I have this resource my_resource, that takes a list of object as attribute.

resource "my_resource" "dummy" {
  objects = [
    {
      field_1 = false
      field_2 = "value_1"
    },
    {
      field_1 = true
      field_2 = "value_2"
    },
    {
      field_1 = false
      field_2 = "value_3"
    }
  ]
}

My issue is that the API to read the resource reorders the list of objects (objects with field_1 = true are placed first in response).

Thus, when applying the given plan, the provider will produce the following error :

Error: Provider produced inconsistent result after apply

When applying changes to my_resource.dummy, provider
"provider[\"registry.terraform.io/hashicorp/my_resource\"]" produced an
unexpected new value: .objects[0].field_1: was cty.True, but now
cty.False.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

How can I make this work ? (I would like the plan to work regardless of the order of objects)

Solution

If your remote API does not treat the order of the items as meaningful/persistent then Terraform's list data type is probably not the correct type to use to model this data.

There are two other options that might work better.

If one of the two attributes you've shown is considered by the remote system to be a unique identifier for an item then you could use a map of objects where that unique identifier is used as the key instead of part of the value.

For example, if your field_2 were expected to be unique across all of the items then you could take its value from the map keys instead and then the user would specify the argument like this:
```
objects = {
  value_1 = {
    field_1 = false
  }
  value_2 = {
    field_1 = false
  }
  value_3 = {
    field_1 = false
  }
}
```
This approach would not be appropriate if none of your attributes are considered by the remote system to be a unique identifier for the object, because it would be impossible for two objects to have the same value of field_2.
If none of the attributes alone is a unique identifier, but the remote API requires all of the objects to be unique as a whole, then you could declare it as a set of objects, which is Terraform's data type for unordered collections of unique values.

The syntax for declaring it in the configuration would be the same as your example in this case, because Terraform can automatically convert from a tuple (which is what the [ ... ] syntax generates) into a set.

If your remote API does not preserve order but yet it doesn't require any sort of uniqueness across the objects then unfortunately there isn't really a good mapping of that API design into Terraform. Terraform wants to be able to distinguish between an object edited in place vs. an object being removed and another one being added, and so it needs at least some way to track the identity of each object from one plan/apply round to the next.

However, there is a more complicated answer you could use as a last resort of none of Terraform's other data types are a better fit for your API: you can write custom logic to check whether the result is consistent, rather than returning the final data to Terraform and having Terraform check whether it's consistent (which it isn't in your case, so you got an error.)

I assume that in your "Create" and "Update" implementations currently you have some logic that takes the response from the API call to create or update the object and uses that to construct the value for objects that you return to Terraform. That is a typical design that works when the API's behavior is consistent with Terraform's expectations for the data types you've used to model the API, but it doesn't work if the API doesn't conform to Terraform's assumptions.

Instead then, you can write a function which takes two values for objects and returns true if the remote API would consider them equivalent. Let's call it objectsAreEquivalent for the sake of this reply.

After you make your API call to create or update, you'd construct a new objects value just as you currently do, but instead of just returning it immediately to Terraform instead you would pass it to your objectsAreEquivalent function along with the value Terraform had provided in req.Plan.

If objectsAreEquivalent returns true then you can discard the new object you constructed and just return the value from req.Plan verbatim. Terraform will then find that it matches the original value, and so it will pass the consistency check due to the order being preserved.

If objectsAreEquivalent returns false then presumably your API has a bug, because it's returned something different than what the module author declared. In that case you might choose to return your own error message saying that the API returned an invalid result.

There isn't really anything useful your provider can do if the remote API did something different than it was told to do... that is exactly the sort of consistency problem that the Terraform error message you saw is attempting to catch. But if you return your own error message instead then you can at least blame the remote API for the problem instead of Terraform blaming your provider, which will then hopefully cause the bug reports to be sent to a more useful place.