nifi 3 node docker + ssl getting Untrusted proxy CN=localhost, OU=NIFI
Followed detailed instructions to set up a 3 node docker hosted nifi cluster + ssl with standalone certificate. Steps that are taken:
- set up docker to start up persistent 3 node nifi cluster (nifi01,nifi02,nifi03,registry and nifi_zookeeper)
- since they all are in docker bridged network, hostnames are set as above. However in order to access UI, I used ssh tunnel to proxy as "localhost:8443:remotehost:6950" . so I can access them from browser as "https://localhost:8443/nifi".
- Then set up oidc for which need ssl . so generated cert and kept it in shared location.
./bin/tls-toolkit.sh standalone -n localhost --subjectAlternativeNames 'localhost,0.0.0.0,nifi01,nifi02,nifi03,nifi_registry'
- followed : https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#cluster-node-identities and set up authorizers.xml which created users.xml /authorizations.xml as below
authorizers.xml
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">xyz@oidc.domain.com</property>
<property name="Initial User Identity 2">CN=nifi01, OU=NIFI</property>
<property name="Initial User Identity 3">CN=nifi02, OU=NIFI</property>
<property name="Initial User Identity 4">CN=nifi03, OU=NIFI</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">xyz@oidc.domain.com</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1">CN=nifi01, OU=NIFI</property>
<property name="Node Identity 2">CN=nifi02, OU=NIFI</property>
<property name="Node Identity 3">CN=nifi03, OU=NIFI</property>
<property name="Node Group"></property>
</accessPolicyProvider>
<authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
which generated users.xml and authorizations.xml as : users.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<tenants>
<groups/>
<users>
<user identifier="c3f09c4d-0c92-3f45-b2e4-877def626d99" identity="CN=nifi02, OU=NIFI"/>
<user identifier="882592de-bfd2-35fa-b447-e6866f066684" identity="CN=nifi01, OU=NIFI"/>
<user identifier="bae641ef-8e45-3838-8dde-8e012ada53f5" identity="CN=nifi03, OU=NIFI"/>
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814" identity="xyz@oidc.domain.com"/>
</users>
</tenants>
authorizations.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<authorizations>
<policies>
<policy identifier="f99bccd1-a30e-3e4a-98a2-dbc708edc67f" resource="/flow" action="R">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="4962996d-5f2e-3c80-94b3-1eaa2d71cedc" resource="/data/process-groups/2fa42d02-b336-30cd-aa96-f1aea310af6f" action="R">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
<user identifier="c3f09c4d-0c92-3f45-b2e4-877def626d99"/>
<user identifier="882592de-bfd2-35fa-b447-e6866f066684"/>
<user identifier="bae641ef-8e45-3838-8dde-8e012ada53f5"/>
</policy>
<policy identifier="cf5bec73-a284-3fae-811c-3e40e3db25e6" resource="/data/process-groups/2fa42d02-b336-30cd-aa96-f1aea310af6f" action="W">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
<user identifier="c3f09c4d-0c92-3f45-b2e4-877def626d99"/>
<user identifier="882592de-bfd2-35fa-b447-e6866f066684"/>
<user identifier="bae641ef-8e45-3838-8dde-8e012ada53f5"/>
</policy>
<policy identifier="1e6048dc-8ba2-34ee-a641-a1e260c55d75" resource="/process-groups/2fa42d02-b336-30cd-aa96-f1aea310af6f" action="R">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="92d7b372-f63a-30ab-a107-f70ea0bbc8d9" resource="/process-groups/2fa42d02-b336-30cd-aa96-f1aea310af6f" action="W">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="b8775bd4-704a-34c6-987b-84f2daf7a515" resource="/restricted-components" action="W">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="627410be-1717-35b4-a06f-e9362b89e0b7" resource="/tenants" action="R">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="15e4e0bd-cb28-34fd-8587-f8d15162cba5" resource="/tenants" action="W">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="ff96062a-fa99-36dc-9942-0f6442ae7212" resource="/policies" action="R">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="ad99ea98-3af6-3561-ae27-5bf09e1d969d" resource="/policies" action="W">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="2e1015cb-0fed-3005-8e0d-722311f21a03" resource="/controller" action="R">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="c6322e6c-4cc1-3bcc-91b3-2ed2111674cf" resource="/controller" action="W">
<user identifier="52b536d4-50ee-351a-9f9a-1f9ee1501814"/>
</policy>
<policy identifier="287edf48-da72-359b-8f61-da5d4c45a270" resource="/proxy" action="W">
<user identifier="c3f09c4d-0c92-3f45-b2e4-877def626d99"/>
<user identifier="882592de-bfd2-35fa-b447-e6866f066684"/>
<user identifier="bae641ef-8e45-3838-8dde-8e012ada53f5"/>
</policy>
</policies>
</authorizations>
oidc is working as expected and authentication is working via SSO. however, once authenticated I am getting below error on UI.
Those look like cert issued authority from toolkit. How to fix this? I tried several options and articles and none seem to work. The above authorizers.xml works because service is working. Otherwise, getting "unable to seed policy for" error.
Can anyone help?
Update: Checked nifi access log and found below:
2022-09-08 15:40:56,879 INFO [main] o.a.n.a.single.user.SingleUserAuthorizer Initializing Authorizer
2022-09-08 15:40:56,986 INFO [main] o.a.n.a.FileUserGroupProvider Users/Groups file loaded at Thu Sep 08 15:40:56 UTC 2022
2022-09-08 15:40:56,987 INFO [main] o.a.n.a.FileAccessPolicyProvider Added mapped node CN=nifi01, OU=NIFI (raw node identity CN=nifi01, OU=NIFI)
2022-09-08 15:40:56,987 INFO [main] o.a.n.a.FileAccessPolicyProvider Added mapped node CN=nifi03, OU=NIFI (raw node identity CN=nifi03, OU=NIFI)
2022-09-08 15:40:56,987 INFO [main] o.a.n.a.FileAccessPolicyProvider Added mapped node CN=nifi02, OU=NIFI (raw node identity CN=nifi02, OU=NIFI)
2022-09-08 15:40:57,000 INFO [main] o.a.n.a.FileAccessPolicyProvider Authorizations file loaded at Thu Sep 08 15:40:57 UTC 2022
2022-09-08 15:40:57,001 INFO [main] o.a.n.a.single.user.SingleUserAuthorizer Configuring Authorizer
2022-09-08 15:41:03,346 INFO [main] o.a.n.w.s.o.StandardOidcIdentityProvider OpenId Connect: Available clientAuthenticationMethods [client_secret_basic, client_secret_post, private_key_jwt, tls_client_auth]
2022-09-08 15:44:20,874 INFO [NiFi Web Server-19] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 172.20.0.1 [<anonymous>] GET https://localhost:8443/nifi-api/flow/current-user
2022-09-08 15:44:20,890 WARN [NiFi Web Server-19] o.a.n.w.s.NiFiAuthenticationFilter Authentication Failed 172.20.0.1 GET https://localhost:8443/nifi-api/flow/current-user [Anonymous authentication has not been configured.]
2022-09-08 15:44:34,264 INFO [NiFi Web Server-23] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 172.20.0.6 [<xyz@oidc.domain.com><CN=localhost, OU=NIFI>] GET https://nifi01:8443/nifi-api/flow/current-user
2022-09-08 15:44:34,275 WARN [NiFi Web Server-23] o.a.n.w.s.NiFiAuthenticationFilter Authentication Failed 172.20.0.6 GET https://nifi01:8443/nifi-api/flow/current-user [Untrusted proxy CN=localhost, OU=NIFI]
2022-09-08 15:46:26,568 INFO [NiFi Web Server-23] o.a.n.w.s.NiFiAuthenticationFilter Authentication Started 172.20.0.6 [<xyz@oidc.domain.com><CN=localhost, OU=NIFI>] GET https://nifi01:8443/nifi-api/flow/current-user
2022-09-08 15:46:26,569 WARN [NiFi Web Server-23] o.a.n.w.s.NiFiAuthenticationFilter Authentication Failed 172.20.0.6 GET https://nifi01:8443/nifi-api/flow/current-user [Untrusted proxy CN=localhost, OU=NIFI]
Looks like "CN=localhost, OU=NIFI" is getting added by the ssl certificate? Maybe someone can explain.
Solution
UPDATE:
Was able to fix the issue by adding Node Identity user to accessPolicyProvider and userGroupProvider for the proxy on authorizers.xml.
userGroupProvider:
<property name="Initial User Identity 5">CN=localhost, OU=NIFI</property>
accessPolicyProvider:
<property name="Node Identity 4">CN=localhost, OU=NIFI</property>
- Docker Compose wait for container X before starting Y
- How to use rules_oci to run an image local via Docker?
- Getting error "Get http://localhost:9443/metrics: dial tcp 127.0.0.1:9443: connect: connection refused"
- How to install pyarrow on an Alpine Docker image?
- Unable to push image to ACR from terraform - could not get auth config (the credentialhelper did not work or was not found):
- Docker returning Error 133: Failed to create temporary file with plenty of disk space
- Database's backup location that created by Azure Data Studio on Mac M1
- ubuntu wslregisterdistribution failed with error: 0x80041002
- System.InvalidOperationException: Failed to deploy distro docker-desktop to <localpath>: exit code: -1
- Installing specific version of NodeJS and NPM on Alpine docker image
- intellij idea wsl 2 can't load projet from \\WSL.LOCALHOST\[distrib]
- Docker "protocol not available" after context change
- How to configure different dockerfile for development and production
- Is it possible to convert the publish method of an App Service from Code to Docker?
- Jib create folder for temporary files and changing folder ownership
- Heroku: How to release an existing image in gitlab CI/CD?
- Access to full /proc in Docker container
- lookup registry-1.docker.io: no such host
- Nestjs doesn't handle request from nginx proxy_pass using docker-compose
- PackagesNotFoundError : The following packages are not available from current channels:
- Prisma openssl version issue
- why are alpine versions of node throwing an error that @jridgewell/gen-mapping module not found?
- Safe ways to specify postgres parameters for healthchecks in docker compose
- How to get a list of images on docker registry v2
- Docker is in volume in use, but there aren't any Docker containers
- PhpStorm inserts wrong client_host IP when trying to run debug session
- How to Properly Inject Service Account Credentials in Google Cloud Build for Cloud Run Deployment?
- Docker - COPY file from different working directory to another is not working
- starting container process caused: exec: "fastapi": executable file not found in $PATH: unknown
- docker login fails while using docker-credential-secretservice