Mule4 JWT connection to BigQuery fails during build

I am trying to use BigQuery connector 1.1.5 in Mule4 flow. The Idea is

to query the data from salesforce
Load it into stg_account table
Merge it into final table.

My test flow is as follows:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core" xmlns:salesforce="http://www.mulesoft.org/schema/mule/salesforce"
    xmlns:bigquery="http://www.mulesoft.org/schema/mule/bigquery"
    xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/bigquery http://www.mulesoft.org/schema/mule/bigquery/current/mule-bigquery.xsd
http://www.mulesoft.org/schema/mule/salesforce http://www.mulesoft.org/schema/mule/salesforce/current/mule-salesforce.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
    <flow name="sf-test-flowFlow" doc:id="1d44b6b8-d767-4f73-b601-bfd4870c6df1" >
        <http:listener doc:name="account listener" doc:id="a91baf52-7335-49a2-b2a2-a7ed0ce29e11" config-ref="HTTP_Listener_config" path="/account/"/>
        <bigquery:query doc:name="create stg_account table" doc:id="106b2de9-d9dd-4acc-b733-04a811b7aa9f" config-ref="BigQueryConfiguration" target="create_table_output">
            <bigquery:query-values-content ><![CDATA[Should I just write a create table ?
CREATE OR REPLACE TABLE `project.table` (
  etl_id STRING(18) ,
  delete_flag STRING(2) ,
  recursive_flag STRING(5) ,
  id STRING(18) ,
  isdeleted STRING(5) ,
  namex STRING(765) ,
  typex STRING(765) ,
  recordtypeid STRING(18) ,
  parentid STRING(18)
  );]]></bigquery:query-values-content>
        </bigquery:query>
        <salesforce:query-all doc:name="Query account object" doc:id="cbc52837-4d70-45cb-a948-fd4cd2286780" config-ref="Salesforce_sandbox" target="query_account_output">
            <salesforce:salesforce-query ><![CDATA[select 
Id,
IsDeleted,
RecordTypeId,
ParentId
from account]]></salesforce:salesforce-query>
        </salesforce:query-all>
        <set-payload value="#[%dw 2.0&#10;output application/json&#10;---&#10;vars.query_account_output]" doc:name="Set payload with query result" doc:id="b8af5836-730f-4d13-b4c7-31fadd63b0f4" />
        <ee:transform doc:name="Transform payload for Bigquery insert" doc:id="97e3bb52-188b-4ab4-a72e-3c919af7b70f" >
            <ee:message >
                <ee:set-payload ><![CDATA[%dw 2.0
output application/json
fun flatten_address(address):String = address.street default "" ++ (" , " ++ address.city default "") ++ (" , " ++ address.state default "") ++ (" , " ++ address.postalCode default "") ++ (" , " ++ address.country default "")
---
{
  rows : payload map(row , index) -> {
    id : row.Id,
    isdeleted : row.IsDeleted,
    recordtypeid : row.RecordTypeId,
    parentid : row.ParentId,
    billingstreet : row.BillingAddress.street,
    billingcity : row.BillingAddress.city,
    billingstate : row.BillingAddress.state,
    billingpostalcode : row.BillingAddress.postalCode,
    billingcountry : row.BillingAddress.country,
    billingaddress : if(isEmpty(flatten_address(row.BillingAddress))) null else flatten_address(row.BillingAddress),
    shippingstreet : row.ShippingAddress.street
    }
  }
  ]]></ee:set-payload>
            </ee:message>
        </ee:transform>
        <bigquery:insert-job doc:name="Insert into stg_table Job" doc:id="7128c1d4-06b5-4a6d-bc9b-a080d99a5304" config-ref="BigQueryConfiguration" target="insert_output">
            <bigquery:job-values-content ><![CDATA[<not sure how to give the insert statement>]]></bigquery:job-values-content>
        </bigquery:insert-job>
        <bigquery:query doc:name="Merge into account table" doc:id="3ad7331e-fe24-41fe-92aa-89e237431684" config-ref="BigQueryConfiguration" target="account_merge_result">
            <bigquery:query-values-content ><![CDATA[Should I just write a Merge query here?]]></bigquery:query-values-content>
        </bigquery:query>
        <logger level="INFO" doc:name="Logger" doc:id="60c7d95d-b25c-4386-8cc9-a6fc55308a28" message='#[%dw 2.0&#10;output application/json&#10;---&#10;{&#10;  exit_status : if(isEmpty(vars.insert_output.insertErrors)) "Inserted Account records"&#10;  else {&#10;    "error": vars.insert_output.insertErrors&#10;  }&#10;}]'/>
    </flow>
</mule>

and global.xml having connections as follows:

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:bigquery="http://www.mulesoft.org/schema/mule/bigquery" xmlns:http="http://www.mulesoft.org/schema/mule/http"
    xmlns:tls="http://www.mulesoft.org/schema/mule/tls"
    xmlns:kafka="http://www.mulesoft.org/schema/mule/kafka" xmlns:salesforce="http://www.mulesoft.org/schema/mule/salesforce" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/salesforce http://www.mulesoft.org/schema/mule/salesforce/current/mule-salesforce.xsd
http://www.mulesoft.org/schema/mule/kafka http://www.mulesoft.org/schema/mule/kafka/current/mule-kafka.xsd
http://www.mulesoft.org/schema/mule/tls http://www.mulesoft.org/schema/mule/tls/current/mule-tls.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/bigquery http://www.mulesoft.org/schema/mule/bigquery/current/mule-bigquery.xsd">
    <salesforce:sfdc-config name="Salesforce_sandbox" doc:name="Salesforce Config" doc:id="fd9cec1d-ae93-47cb-9268-29511623e974" >
        <salesforce:basic-connection username="..." password="...." securityToken="...." url="https://......com/services/Soap/u/59.0"/>
    </salesforce:sfdc-config>
    <http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="4e55c859-34b9-4c81-b94d-b1ceef77fcd0" basePath="/api" >
        <http:listener-connection host="0.0.0.0" port="8083" />
    </http:listener-config>
    <bigquery:config name="BigQueryConfiguration" doc:name="BigQuery  Configuration" doc:id="093fa199-f59a-492b-99b5-ce524007e538" >
        <bigquery:jwt-connection privateKeyId='"..."' privateKey="-----BEGIN PRIVATE KEY-----.....-----END PRIVATE KEY-----&#10;" issuer="engineering-582218145960@bcs-dfs-eng-sbx.iam.gserviceaccount.com" projectId="bcs-dfs-eng-sbx" />
    </bigquery:config>
</mule>

When I build it, I get the following error

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  24.898 s
[INFO] Finished at: 2024-03-01T14:00:08-06:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.mule.tools.maven:mule-maven-plugin:4.0.0:process-classes (default-process-classes) on project sf-salesforce-producer: Execution default-process-classes of goal org.mule.tools.maven:mule-maven-plugin:4.0.0:process-classes failed: There were '2' errors while parsing the given file 'global.xml'.
[ERROR] Full list:
[ERROR] org.xml.sax.SAXParseException; lineNumber: 18; columnNumber: 2020; cvc-datatype-valid.1.2.3: '-----BEGIN PRIVATE KEY-----
......
[ERROR] -----END PRIVATE KEY-----
[ERROR] ' is not a valid value of union type 'expressionString'.
[ERROR] org.xml.sax.SAXParseException; lineNumber: 18; columnNumber: 2020; cvc-attribute.3: The value '-----BEGIN PRIVATE KEY-----
......
[ERROR] -----END PRIVATE KEY-----
[ERROR] ' of attribute 'privateKey' on element 'bigquery:jwt-connection' is not valid with respect to its type, 'expressionString'.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException

When i do test connection to bigquery it works. But when I build it, it fails. For some reason it looks like it is not taking the private key properly.

The reason for using BigQuery connector 1.1.5 is that the other connector 2.2.0 takes a json file in connector. I cannot encrypt that json file and pass it.

Solution

The error message seems to be clear: the value of the privateKey attribute is not valid. "-----BEGIN PRIVATE KEY-----.....-----END PRIVATE KEY-----
" is not a valid key, but also it is probably not in the format expected.

Try putting a real private key removing all new line characters, in a single line.

Example of the format (not a real key):

privateKey="-----BEGIN PRIVATE KEY-----MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDHf8Cx/XVyZ9ZTtD2ae9hh6dj7bGm4aIoXmbNCAOsocbkNWza2TKTczaE8flvDFhH/+8fj8I6360Znba8TSEyA/+K4Voynylw+GXLNF8Wg9rrcQqdW-----END PRIVATE KEY-----"