gitlabgitlab-cikeycloak

Keycloak 18 in Gitlab Service sometimes does not load realm (without error)


I am wondering if any one would know about this problem: I am starting a Keycloak as a Gitlab service in order to run integration tests in a pipeline, using the "--import-realm" option. It works very well locally, and it works some of the times in Gitlab. However, sometimes (I'd say a little more than 50%), the realm is simply not imported, without any error message (and then of course my test fails).

Here is my job description:

integration-tests-common:
  variables:
      FF_NETWORK_PER_BUILD: "true"  
      KEYCLOAK_DATA_IMPORT_DIR: /builds/js-dev/myproject/Keycloak-testapp/data
      KEYCLOAK_ADMIN: admin
      KEYCLOAK_ADMIN_PASSWORD: admin
      KC_HTTPS_CERTIFICATE_FILE: /opt/keycloak/certificates/keycloak.crt.pem
      KC_HTTPS_CERTIFICATE_KEY_FILE: /opt/keycloak/certificates/keycloak.key.pem
  services:
      #(custom image below is based on quay.io/keycloak/keycloak:18.0.2)
      - name: myinternalrepo/mykeycloakimage:mytag 
        alias: keycloak
        command: ["start-dev","--import-realm", "--health-enabled=true", "--http-port=8089","--log=console,file"]            
  script: 
    # Before E2E tests: First wait for keycloak
    - |
        set -x
        count=0;
        while [ "$(curl -s -o /dev/null -w '%{http_code}' http://keycloak:8089/health )" != "200" ]
        do
          echo "waiting for Keycloak..."
          sleep 1; 
          let count=count+1;
          if [ $count -gt 100 ]
          then
            echo "Keycloak is not starting, exiting"
            exit 1;
          fi
        done
        echo "Keycloak is UP after $count retries"
        set +x      
    #... (the rest is my integration test)

KEYCLOAK_DATA_IMPORT_DIR is used by a custom entrypoint to create a symbolic link to /opt/keycloak/data/import (since I cannot mount a volume for a Gitlab service, as far as I know):

ln -s $KEYCLOAK_DATA_IMPORT_DIR /opt/keycloak/data/import

In working cases, I have this log:

2022-08-02 05:46:14,468 INFO  [org.keycloak.services] (main) KC-SERVICES0050: Initializing master realm
2022-08-02 05:46:19,869 INFO  [org.keycloak.services] (main) KC-SERVICES0004: Imported realm test from file /opt/keycloak/bin/../data/import/realm-export.json.
2022-08-02 05:46:20,232 INFO  [org.keycloak.services] (main) KC-SERVICES0009: Added user 'admin' to realm 'master'

But in other cases, the log shows no error, it continues as if the import option was not given:

2022-08-02 06:04:14,230 INFO  [org.keycloak.services] (main) KC-SERVICES0050: Initializing master realm
2022-08-02 06:04:18,220 INFO  [org.keycloak.services] (main) KC-SERVICES0009: Added user 'admin' to realm 'master'

I have also added an nginx in the keycloak custom image exposing the Keycloak logs (because it's difficult to get full logs from Gitlab services otherwise!), but I couldn't find anything more in them.

I dont't know if this is a problem with my custom entrypoint and the symbolic link, with Keycloak, or related to Gitlab services...all I know is that when it fails, I retry the job, sometime multiples times, and usually it finally works. Any help would be appreciated.


Solution

  • By adding a "ls" in my custom Keycloak image entrypoint, I noticed that the Gitlab project files are not present in the error cases. So this is more a Gitlab services issue than a Keycloak issue.

    In addition, it is not clear from the Gitlab services doc (https://docs.gitlab.com/ee/ci/services/) if they are supposed to access the project files or not. I had assumed so, because I made a test which worked. But finally, the solution was to integrate my realm's file into my base docker image, and not rely on the files from the repo.