I'm trying to use Harbor
registry with SCDF 2.9.1
in microk8s 1.18.20
.
I successfully configure SCDF to retrieve the Docker Labels of my apps by adding this in SCDF server Config Map :
spring:
cloud:
dataflow:
container:
registry-configurations:
harbor:
registry-host: myhost
authorization-type: dockeroauth2
user: myuser
secret: mypwd
disable-ssl-verification: true
extra:
"registryAuthUri" : 'https://myhost/service/token?service=harbor-registry&scope=repository:{repository}:pull'
Then for pulling images I created a secret with this command line (after configuring my local Docker daemon) :
microk8s.kubectl create secret generic harbor-credentials \
--from-file=.dockerconfigjson=/home/myuser/.docker/config.json \
--type=kubernetes.io/dockerconfigjson
And change Skipper and SCDF Config Map to use it with :
spring.cloud.skipper.server.platform.kubernetes.accounts.default.imagePullSecret=harbor-credentials
spring.cloud.dataflow.task.platform.kubernetes.accounts.default.imagePullSecret=harbor-credentials
But when I try to deploy my app in a stream, I get this error when pulling the image :
Head "https://myhost/v2/scdf/myapp/manifests/latest": x509: certificate signed by unknown authority
How and where do I have to configure Harbor self-signed certificate so that SCDF/Skipper can deploy applications ?
Also ideally I would prefer to remove disable-ssl-verification: true
for Docker Labels too
The solution is quite simple : I only had to copy the Harbor ca.crt
file (downloaded from Harbor UI) in the /ets/ssl/certs
directory.
You can also create the secret directly without configuring Docker daemon with something like :
microk8s.kubectl create secret docker-registry harbor-credentials \
--docker-server=hostname \
--docker-username='user' \
--docker-password=pwd
Additionally if you want to use Harbor as a proxy for OCI images from docker.io, you can configure SCDF all at once by adding these properties during installation (after configuring Harbor proxy cache project dockerhub-proxy
of course) :
global.imageRegistry = hostname/dockerhub-proxy
global.imagePullSecrets = [harbor-credentials]
In this case, your user should have rights to pull on every Harbor projects you need.
Then all images will be pulled from Harbor including kafka, skipper, zookeeper, etc ...
EDIT :
I share a more elegant way to configure Harbor credentials through K8S secret : add the secret as a volume like this.
The container registry is automatically configured. If you need to add options, you add it in the config map like this :
spring:
cloud:
dataflow:
container:
registry-configurations:
harbor:
registry-host: myhost
disable-ssl-verification: true
Mapping between the secret and the additional configuration is made based on the registry-host
and the docker-server
property in the secret if I understood it well
All of this can be made at once with the Helm chart (since v5.0.1) with a YAML like that :
server:
configuration:
containerRegistries:
harbor:
registry-host: hostname
disable-ssl-verification: true
extraVolumes:
- name: harbor
secret:
secretName: harbor-credentials
extraVolumeMounts:
- name: harbor
readOnly: true
mountPath: /etc/secrets/harbor
global:
imageRegistry: hostname/dockerhub-proxy
imagePullSecrets: [harbor-credentials]
deployer:
imagePullSecrets: [harbor-credentials]