[SOLVED] Gitlab CI visualize issues on gradle multi-module with dockerd gitlab-runner

Bonjor,

Hi All, hope all good.

From past days, I was setting up the coverage visualization on the multi-module gradle project with the gitlab instance using gitlab-CI and jacoco test reports and cobertura conversion analysis. And need a small assistance in setting up the gitlab-CI for the visualization task.

My current gitlab-ci.yaml file is as below:

# This file is a template, and might need editing before it works on your project.
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Gradle.gitlab-ci.yml

# This is the Gradle build system for JVM applications
# https://gradle.org/
# https://github.com/gradle/gradle

image: gradle:latest

# Disable the Gradle daemon for Continuous Integration servers as correctness
# is usually a priority over speed in CI environments. Using a fresh
# runtime for each build is more reliable since the runtime is completely
# isolated from any previous builds.
variables:
  GRADLE_OPTS: "-Dorg.gradle.daemon=false"

stages:
  - build
  - test
  - visualize

before_script:
  - export GRADLE_USER_HOME=`pwd`/.gradle

cache:
  paths:
    - .gradle/wrapper
    - .gradle/caches

build:
  stage: build
  script: gradle -g /cache --build-cache assemble
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: push
    paths:
      - build
      - .gradle

test:
  stage: test
  script: gradle -g /cache test jacocoTestReport
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull
    paths:
      - build
      - .gradle
  after_script:
    - mkdir build
    - cat reports/tests/test/index.html | grep -o '<tfoot>.*</tfoot>'
  artifacts:
    when: always
    reports:
      junit: "**/build/test-results/test/**/TEST-*.xml"
    # paths:
    #   - "jacoco-aggregate/jacoco.xml"

coverage:
  stage: visualize
  image: registry.gitlab.com/haynes/jacoco2cobertura:1.0.7
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull
    paths:
      - build
      - .gradle
  script:
    # find all modules containing java source files.
    - jacoco_paths=`find * -path "**/src/main/java" -type d | sed -e 's@^@'"$CI_PROJECT_DIR"'/@'`
    # convert report from jacoco to cobertura, using relative project path
    - python /opt/cover2cover.py reports/jacoco.xml $jacoco_paths > $CI_PROJECT_DIR/build/coverage.xml
  needs: ["test", "build"]
  dependencies:
    - test
  artifacts:
    reports:
      cobertura: "$CI_PROJECT_DIR/build/coverage.xml"

The issue I'm getting is that when a CI stage ran, it clears all the previous build dirs, and also not able to pull from the cache, and the pipeline is failing. And was already failed too many times.

My Gitlab-runner is on docker container.

CI console logs:

Running with gitlab-runner 14.5.2 (e91107dd)
  on docker E9Kbrzec
Preparing the "docker" executor
00:06
Using Docker executor with image registry.gitlab.com/haynes/jacoco2cobertura:1.0.7 ...
Pulling docker image registry.gitlab.com/haynes/jacoco2cobertura:1.0.7 ...
Using docker image sha256:8d5913f8379757d81e4ec8a21308162acb805e0e875d1d157702983baa38f5ba for registry.gitlab.com/haynes/jacoco2cobertura:1.0.7 with digest registry.gitlab.com/haynes/jacoco2cobertura@sha256:1b8ee364eba07c074f907c9e28ade778671e437e09b5e4457849979fb4265065 ...
Preparing environment
00:02
Running on runner-e9kbrzec-project-2-concurrent-0 via 0b8d013e9fb0...
Getting source from Git repository
00:02
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/chirag/example/.git/
Checking out 954c86a7 as ci-coverage...
Removing .gradle/
Removing build/
Removing example-api/build/
Removing example-domain/build/
Removing example-services/build/
Removing example-services/data/
Removing example-services/temp/
Removing example-web/build/
Skipping Git submodules setup
Restoring cache
00:02
Checking cache for ci-coverage...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache
Executing "step_script" stage of the job script
00:02
Using docker image sha256:8d5913f8379757d81e4ec8a21308162acb805e0e875d1d157702983baa38f5ba for registry.gitlab.com/haynes/jacoco2cobertura:1.0.7 with digest registry.gitlab.com/haynes/jacoco2cobertura@sha256:1b8ee364eba07c074f907c9e28ade778671e437e09b5e4457849979fb4265065 ...
$ export GRADLE_USER_HOME=`pwd`/.gradle
$ jacoco_paths=`find * -path "**/src/main/java" -type d | sed -e 's@^@'"$CI_PROJECT_DIR"'/@'`
$ python /opt/cover2cover.py reports/jacoco.xml $jacoco_paths > $CI_PROJECT_DIR/build/coverage.xml
Traceback (most recent call last):
  File "/opt/cover2cover.py", line 163, in <module>
    jacoco2cobertura(filename, source_roots)
  File "/opt/cover2cover.py", line 147, in jacoco2cobertura
    tree = ET.parse(filename)
  File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 1202, in parse
    tree.parse(source, parser)
  File "/usr/local/lib/python3.8/xml/etree/ElementTree.py", line 584, in parse
    source = open(source, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'reports/jacoco.xml'
Uploading artifacts for failed job
00:02
Uploading artifacts...
/builds/chirag/example/build/coverage.xml: found 1 matching files and directories 
Uploading artifacts as "cobertura" to coordinator... ok  id=138 responseStatus=201 Created token=jCWt186i
ERROR: Job failed: exit code 1

Also logs shows that the cache was restored, but writing ls on script only showed the src folder.

Please assist me in setting up the coverage job.

Thanks in advance.