postgresqlquarkusflyway

Vanished checksum difference, changed migration folder checking after quarkus/flyway/postgres upgrade

The riddle/problem Before our updates mentioned below, our google hosted prod deployment showed stable differences in behavior to self hosted staging system

  1. it did not take migrations from "dev" folder into account despite "flyway.locations" was set to look into "common" and "dev" and this value was not explicitly overridden anywhere.
  2. it always calculated different checksums for the same migrations files in comparision to all other modes (staging, dev, test,...). But those differences were stable.

After our updates mentioned below, prod now behave like all other modes and is therefore obviously "unhappy" with the new checksums and those old dev migrations now chipping in out-of-order.

Context data:

We recently updated following possibly related dependencies:

before: id "org.flywaydb.flyway" version "9.4.0"
after: id "org.flywaydb.flyway" version "11.3.0"
before: id 'com.avast.gradle.docker-compose' version "0.16.8"
after: id 'com.avast.gradle.docker-compose' version "0.17.12"
before: org.postgresql:postgresql:42.4.2
after: org.postgresql:postgresql:42.7.4
before: quarkusPluginVersion=2.11.3.Final
after: quarkusPluginVersion=3.18.2
new: id 'org.flywaydb:flyway-database-postgresql:11.3.0'

Best thanks to everyone who read until here :)

The solution seems at first glance obvious:

  1. adapt checksums in prod's flyway_schema_history (either with flyway repair or manually)
  2. add an overriding setting for prod to exclude "dev" folder migrations.

But we don't feel very good applying this fix without understanding the root cause for the differing behavior before.

So, finally the question(s):

  1. Is our fixing approach correct?
  2. Is someone able to explain the differing behavior from before the updates?
Solution

  • With such a jump in product versions it would be difficult to narrow down a specific change that has caused the differences that you are seeing.

    I'm not sure why checksums would have changed - it is the CRC32 of the file so should be consistent between runs and across machines. Likewise, I'm unsure why on one machine the dev folder wasn't being picked up despite being specified

    It does sound like an annoying issue has been resolved though, in that the location was configured but not honored as well as the checksums being different.

    I think that, if you are sure the migrations haven't changed then you should run repair to get the new checksums into the Schema History Table.

    You can also set locations for a specific environment using environment overrides: https://documentation.red-gate.com/fd/environment-flyway-namespace-277578926.html

    For disclosure, I'm a developer working for Redgate on Flyway