javamigrationapache-nifiapache-nifi-toolkit

How to migrate nif flowfile from 1.12 version to 1.16.3


I have a dataflow running in NiFi 1.12.0, the relevant properties from this installation is here:

nifi.sensitive.props.key=
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=

I am facing migration issue in nifi when I upgrade base version to 1.16.3 from 1.12.0. which having following properties.

nifi.sensitive.props.key=testPassword
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=NIFI_ARGON2_AES_GCM_256
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=

I am getting following exception while I execute command for migrating flow file.

nifi@nifi-service-0:/opt/nifi/nifi-toolkit-current/bin$ ./encrypt-config.sh -n $NIFI_HOME/conf/nifi.properties  -f /opt/nifi/data/old_flow.xml.gz -s testPassword -x
[main] WARN org.apache.nifi.properties.ConfigEncryptionTool - The source nifi.properties and destination nifi.properties are identical [/opt/nifi/nifi-current/conf/nifi.properties] so the original will be overwritten
[main] WARN org.apache.nifi.properties.ConfigEncryptionTool - The source flow.xml.gz and destination flow.xml.gz are identical [/opt/nifi/data/old_flow.xml.gz] so the original will be overwritten
[main] WARN org.apache.nifi.properties.AbstractBootstrapPropertiesLoader - System Property [nifi.properties.file.path] not found: Using Relative Path [conf/nifi.properties]
[main] INFO org.apache.nifi.properties.NiFiPropertiesLoader - Loading Application Properties [/opt/nifi/nifi-current/conf/nifi.properties]
[main] INFO org.apache.nifi.properties.NiFiPropertiesLoader - Loading Application Properties [/opt/nifi/nifi-current/conf/nifi.properties]
[main] INFO org.apache.nifi.properties.ConfigEncryptionTool - Loaded NiFiProperties instance with 138 properties
[main] INFO org.apache.nifi.properties.NiFiPropertiesLoader - Loading Application Properties [/opt/nifi/nifi-current/conf/nifi.properties]
[main] INFO org.apache.nifi.properties.ConfigEncryptionTool - Migrating flow.xml file at /opt/nifi/data/old_flow.xml.gz. This could take a while if the flow XML is very large.
[main] ERROR org.apache.nifi.properties.ConfigEncryptionTool - Encountered an error: Decryption Failed with Algorithm [AES/GCM/NoPadding]
Encountered an error migrating flow content



usage: org.apache.nifi.properties.ConfigEncryptionTool [-h] [-v] [-n <file>] [-o <file>] [-l <file>] [-i <file>] [-a <file>] [-u <file>] [-f <file>] [-g <file>]
       [-b <file>] [-S <protectionScheme>] [-k <keyhex>] [-e <keyhex>] [-H <protectionScheme>] [-p <password>] [-w <password>] [-r] [-m] [-x] [-s
       <password|keyhex>] [-A <algorithm>] [-P <algorithm>] [-c]



This tool reads from a nifi.properties and/or login-identity-providers.xml file with plain sensitive configuration values, prompts the user for a root key, and
encrypts each value. It will replace the plain value with the protected value in the same file (or write to a new file if specified). It can also be used to
migrate already-encrypted values in those files or in flow.xml.gz to be encrypted with a new key.

Please help to solve this issue


Solution

  • Solved by myself.

    Issue can be resolved by following steps

    1. Before migration if you don't have nifi.sensitive.props.key set, set it using following command ${NIFI_TOOLKIT_PAT}/bin/encrypt-config.sh -f /opt/nifi/nifi-current/data/flow.xml.gz -p ${NIFI_HOME}/conf/nifi.properties -s <NEW_KEY_TO_SET> -x
    2. Once key is set upgrade nifi. Since in newer version algorithm is changed set it using command ${NIFI_HOME}/bin/nifi.sh set-sensitive-properties-algorithm <NEW_ALGORITHM>
    3. Once algorithm set, encrypt again using command ${NIFI_TOOLKIT_PAT}/bin/encrypt-config.sh -f /opt/nifi/nifi-current/data/flow.xml.gz -p ${NIFI_HOME}/conf/nifi.properties -s <NEW_KEY_TO_SET> -x

    Now you will get all compatible files with respect your latest version