On MarkLogic 10.0-3, using ml-data-hub plugin 5.2.1, I would like to specify variable source and target databases in mapping steps of flows when deploying to various environments (saliman.properties plugin 1.4.6) based on the mlAppName
property, e.g.:
[...snip, snip...]
"2" : {
"name" : "myStep",
"description" : "a step in my flow",
"options" : {
"additionalCollections" : [ ],
"sourceQuery" : "cts.collectionQuery([\"targetCollection\"])",
"mapping" : {
"name" : "MyFlow-myStep",
"version" : 0
},
"targetEntity" : "MyEntity",
"sourceDatabase" : "%%mlAppName%%-STAGING",
"collections" : [ "myStep", "MyEntity" ],
"permissions" : "data-hub-operator,read,data-hub-operator,update",
"validateEntity" : false,
"sourceCollection" : "sourceCollection",
"outputFormat" : "json",
"targetDatabase" : "%%mlAppName%%-FINAL"
},
"customHook" : {
"module" : "",
"parameters" : { },
"user" : "",
"runBefore" : false
},
"retryLimit" : 0,
"batchSize" : 100,
"threadCount" : 4,
"stepDefinitionName" : "entity-services-mapping",
"stepDefinitionType" : "MAPPING"
},
[...snip, snip...]
However, the token substitutions don't happen. Is this impossible, or is there a better best-practice method to implement this? Thanks.
It's unfortunately not yet supported. You'd have to use a custom Gradle task to manipulate the contents of the documents after they've been loaded. Please create a ticket in the DHF github project though to request this feature, it's a worthwhile one to add.